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ABSTRACT 


This report describe;? the work. oerformed durlnn ihe last year 
op a contlnuirq prolect to develop a computer oased consultant (CPC) 
system. The svsterr' Is being designed to talk (in ordinary English) 
with a human user to hein him Perform tasks entailing maintenance and 
troubleshooting of electromechanlea 1 equipment. 

our plan for this research calls for building a series of 
increasingly sophisticated systems to be demonstrated in 1976. 1977, 
and 197fl, nur current (April 1975, demonstration system is « base 
for future systems and already illustrates abilities to interact with 
an apprentice to help him assemble a small air compressor. 

The body of the report is divided into two major sections. 
On"* of these describes the technology behind the April 1975 
demonstration system. The other describes work in Droqress that will 
contribute to the power and versatility of future demonstration 
systems. 


nur 1975 system has the following specific abilities: it can 
generate and execute olans for assembly/disassembly at several levels 
of detail. It can answer queries from the apprentice about the 
status of the equipment, it can point at parts of the compressor and 
can name parts pointed to by the apprentice, it has a rudimentary 
ability for tw 0 -w a y communication usinq speech. The basis for each 
of these is described in detail. 
















boric supporting subsequent systems has been In the area of 


natural language understanding# modeling# troubleshooting# and 


vision. Secarate subsections of the reDort deal with each of these 


areas. 


The rerort also describes progress on some supportlna 


activities including the SRI Artificial Intelligence Center Computer 


Facility, the language GLISP, hardware interface work# and a scanning 


laser range finder. It concludes with a list of presentations and 


publications by the fcrolect staff. 
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INTRODUCTION 


A. T he Computer-Based Consultant Project 

1. Project Goal 

For the past 18 months or so we have been working on 
a svstetn that we call the Computer-Based Consultant fCBC). The 
system will be ablr to talk (in natural languaqe) with a human user 
to help Mm perform tasks in seme particular task environment. We 
intend to build a consultant that approximates the communication, 
perceptual reasoning, and factual knowledge skills of an actual 
expert on the scene. 


Our main anal Is to create the fundamental technoloav 
needed to build such consultant systems, we expect for hope) that a 
good portion Of this technoloav wm be independent of the details of 
the Darticular kind of expertise being dealt with and of the details 
of the particular task environment. Thus we view our work as having 
Potentially hian payoff because of the great variety of applications 
in several task environments in whlcn consulting expertise is needed 
or would be useful . 

?. Background 

The technoloav of computer-based consultants can be 

viewed as stemming from the confluence of two lines of research. One 

line has centered on formulating and encodina a great deal of 

1 
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knowledge at>our a chosen nroMen domain in order to Produce a Proaram 
whoi»e performance rivals expert humans. Often cited examples of this 
research include programs that analyze chemical structure ti,21»« 
perform svrholic integration C 31# or play board qames [4,5,6], 

The second line of research has focussed on methods 
for constructing a proaram that can carry on a dialoa with a user. 
Important contributions to this research have come from work in 
computer aided instruction, and from work in understanding typed and 
spoken natural larouaqe. Representative examples of this work 
include proqrams to carry out a "mixed initiative" tutorial dialoq 
[7,R]; to engaae in a dialoq about a toy block world 19]; and to 
understand SDoken Fnolish sentences about such diverse topics as 
olumbinn [10], news stories [11], moor rocks [12], or submarines 
[ 131 . 


Perhaps one of the best examples to date of a small 
computer-based consultant Is the MYCIN system [14], This system 
provides advice to physicians on the diagnosis and therapy of certain 
classes of bacterial diseases. It solicits various kinds of medical 
data frr a physician user, can answer his questions (expressed in a 
hlohly restricted natural Fnqllsh format), and can accept advice from 
him regarding generally useful rules for diagnosis and therapy. 


♦References are listed at the end of this report 






















3. The Task t-rivironrpnf 

Notwtthstandino our ooal of aevelorino t 'in la nenr a 1 
and widely useful technoloov* ne must , of course* select some 
Particular t^sk e n vironmenf for the CdC svstem. .• e h^v« attornpted to 
select a task environment fbat is an important <inc 1 i cat J on area in 
its own rloht as well as one that serves well as a typical 
representative of a wide varietv of applications. 

Wp h ^ vp delected the task of rePajrinci# moditVlno, 
and checkino nut complex electromechanical eoninnent. our or system 
will he helpinn an inexperienced mechanic Cwhom, *e shall call an 
apprentice) as hp works on a Piece of equipment In a "workstation." 
The kinds of problems encountered uslra this particular task 
environment are typical of those 0 f many other environments, 
furthermore, maintenance of eouipment Is an Important task In itself, 
costina literally billions of dollars each year. 7hus, there ts a 
hiah potential for suhstantial savims derivtno from Improved 
efficiencies in this one application area alone. 

Before dlscussinn tne details and snecltlcations of 
the CHC system itself, let us first briefly consider some of thp 
characteristics of eouioment maintenance as a task area. tmaolne an 
apprentice worklnq on a Diece of eouipment in a workstation like the 
one sketched in Moure 1, Typically he is responsible for a variety 
of lohs, such as troubleshootim, renalrtno, or m 0 difyi no ePuIrment. 






































In ordt>r to do those lobs# he needs certain Vincis of specialised 
Knowiedoe; he must know ano'it the us e nf various tools, about 
principles ot trouh1eshootina , and about principle? ot assembly and 
disassembly, and he mist- also know a certain amount of detail about 
tbp construrtlon ano nDpratlon of the specific equipment on hand. 

6 traditional wav ot ronvevino this k-now]elqe to a 
mechanic has bepn throuoh the use of maeials. A more rearlv Ideal, 
thouob usually impractical, wav wouli be to make an expait human 
mechanic continuously available as i consultant. This expert row id 
identify various corponents, answer specific auestlons about 
equipment details# suanest t r ouh 1 eg hoot ino sentiences# hypothesize 
causes of failure, warn of hazards, and so forth. 

In order to heain to «xo]ore what would he involved 
in rerlacino the human expert bv a compjter^haspd expert# we recorded 
a number of dlaloas between pxpert human mechanics and novice 
mechanics. Tb° dtaloos concern the air comrressor shown in Flourp ?, 
Two excerpts from these dlaloas are presented below. At the time the 
dlaloas were recorded, the pxrert and no/ice warp in different rooms 
and the expert viewed the scene onlv by means of still pictures taker 
throuqh a television camera, (We did this to simulate tne limited 
visual information likeiy to he available to a computer based 
expert.) The first excerot concerns the suhtask of installinn a nump 


DUlleV on the rumn. 














Excerpt l 


Expert: The rum d d u 11 e v should he next. 

Arorentice: Yes ... uo, does the side of the r>urro 

pulley with the holes face awav from 
the pumD or towards it? 

K : Away from the Dump. 

A: a 11 r1oht, 

F: Did you insert the Kev--that is, the half-moon 

shaped piece? 

A: Yes, I did. 

F: de sure you coecE the aiionment of the two 

pulleys before you tighten the set-screws. 


A; 


Yes, T'm just now fiddltno with that 



























A: lightening the Allen s c r e #i nov. 


1 : OK , t nank vou. 


This fragment illustrates several imrortant ahill ties 
of consultants that contrast sharDiv with a static information source 
like a manual. First# notice that a question from the apprentice is 
answered dire c tiv ana in Ms terms. There is no need for him to 
search throuah a pass of information, or to convert information from 
so abstract or "standard" form into a directly usable form. .jotice 
that the exp-rt is checking on oropress bv offering waminos and 
reminders about critical steos. This has the function not onlv of 
mlnlmlzino errors# but also of allowing the expert to k»eo track of 
the oroqress of tne work. The latter function is the hauls for the 
expert's ability to present relevant advice, and to present it in a 
context that is familiar to the apprentice. 

The second dialog excerpt conc°rns the same subtaSK# 
but was carried out with different participants.. Tt offers an 
interesting comrarlson of the different deman's imposed oy different 


skill levels: 




















txcerpt ? 


Fxpert; Install the pullev on the shaft. 

Aoprentlce: What Is the first thlna to do in 

installing th<* nulley? 

F: Rotate the shaft so that the slot (kevwav) is 

on the top, 

A: OK ... now what? 

F: Place the key in the slot. 

A: Flat siae unw fl rd? 

Fj Yes. 


This short fraament drnmat1cally illustrates "he 
ability of the expert to descend into detailed instructions in order 


























to help a very naive user. Tn<s apprentice neeos much more hel.j than 
the first one did, a situation foreshadnwed bv nls initial question 
about a relatively simple operation. 

T bese short dla.loa excerrts exemplifv some of the 
abilities that a consultant repds in order to bp helpful to the 
aoDr ent Ice mechanic. both Introspection and ntotocol experiments 
Doint out a i,umber of other r e nuir P H abilities, afrnna which are the 
ability to provide advice about trouoleshootlmj to describe the use 
of tools: to describe the appearance of tools for to he able to point 
them ou11 • and, of course, the ability to use lanauaoe. (We shall 
have mope to say In a later section about how ye arp usino protocols 
of this sort to desion the subsystem for orocessino natural 
lanauaoe. 1 

hurinq the Dast year we have been usino the simple 
air compressor of Moure 2 as an exemplar oiece of equipment. While 
the device appears to be reasonably simple, certain of its 
suhaisemhiies are rather complex. For example, the air pumn, shown 
in Moure 3, contains a laroe number of Parts put tooether in complex 
fashion. 

The compressor has served us well as a heoinninq 
device, but we are now ready to move on to some more complex devices. 
We are especially anxious to select a dpvlce demandina rather 
sophisticated troubleshootino skills. Je expect to make a decision 
soon about which new device to use. Several candidates are discussed 
in the Appendix. 
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4. Prelect Hra^nizition an i v, ethod of Approach 

ic p have alreaiv statpd that our noal for the CBC 
orolprt is to build a system that approximates a hurmn expert . T'r Is 
rather difficult for us now to predict orectseiv fust how flop or 
coarsp this aocrox imat ion win hp a - anv aiven future time. In our 
early plans for the rro1*ct 11 5 1 *'e stated some uereral 
specificat1ons for a system we believed would hp demonstrahle in 
197R. Current predress on the nrolecf reinforces our confidence in 
beina ahle to achieve ar, impressive svstem nv that time. 

In Plannina our work- we have relied heavilv on 
studvina tape-recoroed oialoas between nnnan experts and apprentices. 
From these w e have extracted various requirements for our 107 ^ 
demonstration svstem. In summary th°se are: 

fl) The ystpm will ne abl' to Plan looical sentiences 
of actions to accomplish aoals. These rioals will 
include a ranae of tasks involvinq check-out. 
maintenance, trouhleshootInn# repair, and 
assembly^disassemb1v of equipment. 

(2) These sequences of actions win he piannpd to 
whatever level of detail Is appropriate to the 
apprentice's ability and to the other requirements 


o * the system 

















































research questions into four catenorles: nrobl em-sol v 1 nq# natural 
language, vision# anO system intearatinn. Fach cateaory has Its own 
t»am of researchers# although there is considerable overlan and 
sharino of personnel between each of these tasks. nur strategy Is 
that as research progresses on the first three of these topics# the 
interim results will Immediately be Integrated Into a qrowjno 
lemonstrat. 1 on system. Thus# at any time# the current version of the 
demonstration system can be taken as a measure of pronress toward our 
1978 goals. 

The arowlna demonstration svtem *111 be an important 
tactic to ensure proper communication between various parts of the 
orolect. we are also using It to structure malor subgoals and 
scheduling. We have defined certain subsidiary abilities that should 
be achieved by each of three malor demonstrations to occur each April 
in 1975, 197b, and 1977 Mf1. 

A description of the abilities of the CRC system at 
the time this report went to press Is given in the following 
Daraaraohs, Subsequent sections of the report will describe the 
details of the technology underlying this oresent (April 1975) system 
and the work being done in support of future systems. 




















B. 


The 1°7s Demonstration System 


1 . 

that exhibits 
describing the 

workstatlon. 
containing trie 


Genera 1 

In this section we shall describe a demonstration 

the current abilities of the svstem. we heotn bv 
setting of the demonstration. 

The place where the action takes Dlace is called a 
It is a room, aporoxjmate 1y 11 feet by ifl feet in si^e 
following Items: 

A workbench with a tool box and tools. 

A round table with a turntable too on which is 

Placed a small air compressor: this is located in 
the middle of the room with access from all sides. 

A commuter terminal. 

A microphone headset with a long cord that will 
reach to any point in the room. 

A sneaker/amp 11 fler . 

A TV camera, mounted near thp ceilina on a movable 
pan/tllt head. 

A TV diSDlaV (RAMTEK) on which is displayed a TV 

picture of the air compressor, with a superimposed 
line drawing of the air compressor. 

A laser/rangefinder mounted near the TV camera. 

The air compressor is in a Partially disassembled 
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cond111 or 


The belt housinq cover and he]t are removed and lyina on 


the workbench. The pnun holts are removed, and the nump is turned 
awav from Its normal orientation. 

The apprentice enters the room and walks to the 
terminal and finds the fn]]o*ioq messaie: "Your qoal is 'AssenMed 
A Ircomnressor '. Please out on the headset, and sav 'Heady* Into the 
microphone. The computer win nlve vou verbal Instructions. You can 
say any of the followino responses: 

UK indicates vou can do the task. 

HdW Indicates vou need more -retailed Instructions. 

WHY indicates vou want to know the motivation for the 
particular Instruction lust received. 

HUH or WHAT or pLFASK-pepeat will cause the last 
command to re repeated. 

WHEHF is the ... or SHOW mk THE ... followed by the 

name of a component wm result In Positionlnd 
of the laser ranaefinder beam on that component. 

WHAT IS THIS, coupler) with Vour toUchlnq a component 
with the lighted wand, will aet vcn the name 
of the component. 

BREAK, PAUSE, or WAIT will cause an interrupt in the 

proaram execution so that vou mav use the terminal 
to query the nronram about the state of thlnos. 
(Sorry, there is no voice I/O for these nueriesi. 

LOCATION OF ... followed bv the name of a component will 
result in positionina of the laser beam or the 
location where the component is to he positioned. 





















CALJbhATF irvokps a calibration spo'ipncp tor c a m p r a , 
laser, and ranqej- . 

FIX’D fONpuKSSop crisps thp laser/ranqer to determine the 
location and orientation of the corrrpssor and 
update Its ’'O'iPis and displays nccordimiv. 

Hi-LP •'’pans th fl t somethin'? has none wronn. »p w { ] i 

examine the priced inn steps and net you an pxnert 
person it necessary. 

Please adjust the headset and proceed when von are 

ready." 


Ona]1 tatlve Anilities of tnp Computer based Consultant 

The abilities of the CF'C are aiiudPd to in the list 
of responses alven above, and tore details will he presented in 
Section II. In summary, the CBC contains a system for Dlannlnq 
assemhly or disassembly of the atr compressor. The plan Is 
represented by a structure cal 1 a d a procedural npt which# in this 
demonstration, is built from a Drespecified disassembled state of the 
compressoi, namely the one described above. Facb step of the 
assembly plan can be olven at several different levels of detail, 
depend 1na on the needs of the Particular apprentice. The nroqram 
keens an Internal model of the "connectedness" of the different 
components of the air compressor. It also contains a qranhical 
(absolute positions) model of the locations of the different 
components. At the present time these m, 0 aei s are limited. The 


qr aphical me del "knows about" the numn, ^otor. Pressure switch, celt 


























housing frarp, belt houslm rov^r» pump nullev, notor oullev, 
Platform, tar^# and table. The connectivity model "knows about" all 
the visible components, but the two models do not as yet interact 
with each other. 


The ChC !s able to aive responses uslno the voire of 
the Votrax Phoneme svnthesizpr (described in detail in Section ri.F.2 
below! and can understand sooken instructions uslno the VIP-ino 
Phrase recoonlzer (described in Section TT.F.i). The TV camera is 
used in obt.alnlna the location of the brightest point in its field of 
view. This is useful in cooperation with the laser beam, both in 
calibratina the Graphical system and also In ansv-erina questions 
about names of particular parts ot the compressor. The laser beam is 
orooram Controlled, and can bp moved to Point at a aiven comnonent, 
or to indicate calibration points. 



bhen 

these 

various 

ah jlities 

are combined, they 

produce a 

system that 

is 

aireadv 

rather 

unique and impressive. 

Further 

deve1opment 

1 s 

needed 

for 

each 

abilitv, and the 


system-hui1dina task is an onqolm one (see Section II.F). other 
abilities vi11 be added as thev become avallahjp. In particular 
these are a natural lanouaae capability (see Section TI1.B) and a use 
of vision (srpne anaiysisl to aid the problem soivino (see Section 


III.D) 
























3 . 


Fxamc 1 e of a CBC/Apor o nMce Interaction 


The following simulated lUlon between an exnert (E) 
ani an apprentice (A ) shows toe abilities 01 the system as nf a o r 1 1 
The interaction is by voire, with use of comm«»rclallv 
available devices for voice recoonltlon ani voice svnthesis. 

E : THY TO ACHIEVE A S S F M R L F P A l pfOMpHFSSOp 
A: HOW? 

Ej THF. TOP I.E'VEL PI AM Is INSTALL PU P, INSTALL pitupnj, j\f F , 

INSTALL PUMFPUIiL FY, INSTALL RFLT, AND INSTALL iJKLTHWJS TNGCOV KP 
E: HEAPY FOR GUFSTTHNS 
A: show me: the: pump 
(L aser bean moves to point at pumo) 

E: HFADY 

A; .SHOW MK THE: motor 

(Laser be air moves to noint at motor) 

F.j READY 

A; WHF.RF TS THF PE LTE'OtJS 1 \JGKR A MR? 

(Laser beam moves to ooint at the helthousincjframe) 

Ki RFADY 

A* WHAT 15 THIS? (Touchinn tank Platform with wand) 

E: T A MK PLATFORM 

E: HFADY 

A: WHAT Ts TPTS? (Pojntinn with wand at nressnre switch) 

E: PRFSSURF SWITCH CPVFP 
E: RFADY 



















E: INSTALL PUMP 


MOW? 

CONNECT PUMP TO PLATFORM 
HOW? 

POSITION' puvp ON PLATFORM 
LOCATION OF PUMP? 


(Laser heap moves to point at numnrlace on Dlatfor m ) 


A: ok 

Fj POSITION PUMP OU PLATFORM 
A: how? 

E: PLACE THE RlIMp Oh THE PLATFORM TM SUCH A WAY THAT HOLES 
IN THF Plivp ARF MATCHED UP WITH HOLES IN TMF PLATFORM 
At OK 

F: ATTACH pilMp TO PLATFORM 
At HOW? 

E; USE HOLTS TO ATTACH PUMP TO PLATFORM 


A; HOW'? 

E; OFT 4 (1NE AND A QUARTER INCH « S/16 HEXAGONAL HOLTS AND 

4 5 /1 6 HOLE DIAMETER FLAT WASHERS AND 4 STZK P/16 HEXAGONAL NlJTS 
At HUH? 

(Apprentice did not hear the command) 

E: GF.T 4 ONE AND A QUARTER INCH « S /1 6 MFXAGONAL HOLTS AND 

4 S/16 HOLE DIAMETER FLAT WASHERS AND 4 SIZE P/16 HEXAGONAL NUTS 
A: HOW? 

Et GFT l ONE AND A QUARTER INCH # S/J6 HEXAGONAL HOLT 
A; ALL RIGHT 

E: please rephrase your answfp 
(F xoert did not recoonize the r*>soonse) 


» % v.'*.. 
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A 

E 

A 

E 

A 

E 

A 

E 

A 

IT 

A 

E 


A 

E 

A 

E 

A 

E 

A 

E 

A 

E 

A 

E 

A 


OK 

GFT ONE S /16 HOF E DIAMETER ELAT wASHER 
OK 

GET on: SI 7 .F Q/ 1 fc HE X a COM a t. ‘I U T 
OK 

GET 3 ONE AND A OUARTFP iNCH « S/lf HEXAGONAL DOLTS AND 
3 5/16 HOLE DIAMETER FLAT WASHERS A N i) \ S T 7 . F H/ 1 A HEXAGON A I, NUT, 
OK 

LOOSEI.y FASTEN THE 4 HOLTS 
HOW? 

START IN P*'F HOI T 
HOW? 

PUT THF ROOT THROUGH THE HOLE IN ptIMphASE AMD PLATFORM AND 
WASHEP AND TURN THE NUT CLOCKWISE ONTO THF BOLT AS FAR 
AS YOU CAN 
OK 

LOOSELY FASTEN THF OTHER 3 HOLTS 
OK 

TIGHTEN THF 4 HOLTS 

Hnw? 

tightfn one HOLT 

HOW? 

FIND A SI 7 F Q/is WRENCH TO FIT THF NUT 
OK 

FIND A SIZE I/? TfiOH WRENCH TO FIT THF BOLT HFAD 
OK 

USF THE FIRST riPFNCH TO HOLD THE NUT FIRMLY 

OK 
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E! 
A; 
E: 
A: 
E: 
A: 
E: 
A: 
E ; 
A: 
E: 
A * 
E: 
A: 
E: 
A: 
E: 
A: 
E: 
A: 
E: 

A: 
E: 
A: 
E: 
A 


"SE THE SECOND WRENCH TO TURN THE BOLT C LOCK a I St' U^TH FAlPl.Y TIGHT 
OK 

TIGHTEN THE 11 TMF H 3 HOLTS 
OK 

INSTALL PUMPBPACK 
HP w ? 

CONNECT PU M PBRACE TO BELTHOMSINGFR \t*V 
HOW? 

POSITION PLIMPPRACE ON BF LTHOUSINGFrAMK AND POSITION PUMPBRACE ON PUMP 
OK 

ATTACH PUMPHRACF TO HELTHOUSINGERAME 
HOW? 

USE SCRFWS TO ATTACH piJMpHRACE TO HELTH 0 U$lNGEPAME 
HOW? 

OFT 2 HALF INCH » 14 SCREWS 
UK 

LOOSELY FASTEN THE ? SCREWS 
HOW? 

STAPT IN ONE SCREW 
HOW? 

STAPT IN THE SCREW BY TURNING IT CLOCKWISE AS MUCH AS YOU CAN UNTIL 
IT HOLDS THERE BY ITSELF 
OK 

LOOSELY FASTEN THE OTHER SCREW 
OK 

TIGHTEN THE 2 SCREWS 
HOW? 

TIGHTFN ONE SCREW 





























E: F1MP A SCPKWnpiVF.R that will Fir TMTH TMF JI1TC I UK THE SCREW 
A: why? 

E: IN ORDER TO TIGHTEN THF 1 SCRFW 
A: WHY? 

E; IN nppFP in USE SCRFwS TP ATTACH PilMpRqACF TP BFLTHOUS INGFRAM£ 
WHY? 

in order to attach pimprhace to rflihchisingfrawf 

WHY? 

IN ORDER TO CONNECT PU M PRRACF TO RELTHOUSTNGFRAMF 
WHY? 

IN ORDER Tn INSTAI L Pl.l^DHRACE 
WHY? 

NEXT STEPS APE TO INSTALL PUMPPtJLLEY AND INSTALL FELT AND 
INSTALL BFLTHPI!SINGC0’'ER 
WHY? 

tn order to try to achieve assembled aircomprfssop 

WHY? 

ASK THF' ROSS 
OK 

FIND A SCREWDRIVER THAT WILL, FIT iMTn THF NOTCH of THE SCREW 
OK 

USE THE SCREWDRIVER to TURN THE SCREW CLOCKWISE UNTIL TIGHT 
OK 

TIGHTEN THE Other SCREW 
OK 

CONNECT PUMPER ACE TO PUMP 
OK 
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CONNECT PUMPPUL Lt'Y TC 1 PU M F 


■% 

> • 

■* • 


A ! 

E 

A 

E 

A 

E 

A 

E 

A 

E 

ft 

E 

A 

E 

A 

E 

A 


HOW? 

POSITION PUMPPULLFY ON PMMp 
HOi*? 

pi ACE THF PUMPPIILLEY ON THE PIJMp SHAFT IN SUCH A WAY THAT THF FLAT 
SIDE OF THE SHAFT IS ALIGNED WITH THE FLAT PAp-j INSIDF THE CENTER 
HOLF OF THF PUMpplILLEY A NP THEN PUSH THE PUMPPULLFY ONTO THE PIJ M P 
SHAFT AS FAP AS YOU CAN 
OK 

ATTACH PUMPPULLFY TU PUMP 

new? 

ATTACH THE PUMPPULLFY TO THF, SHAFT OF THF PUMP 
HO VI? 

GET 7 OUAPTFP TNCH * 1/4 SETSCREWS 
OK 

LOOSELY FASTEN THF 2 SETSCREWS 
HOW? 

START IN ONE SETSCREW 
HOW? 

START IN THE SETSCREW BY TURNING IT CLOCKWISE AS MUCH AS YOU CAN 
UNTIL IT HOLDS THFHE BY ITSELF 
OK 

LOOSFIY FASTEN THF OTHFR SFTSCRFW 
OK 

TIGHTEN THE 2 SETSCREWS 

HOW? 24 
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E: 
A: 
E: 
A: 
E: 
A: 
E: 


A: 
E: 
A: 


TIGHTEN nrifcl SF1SCREW 
HOW? 

FIND A SIZE 1/P ALTFUWRENCH 
OK 

TORN THE SFTsrpF.W rLOCKWlS p UNTIL TIGHT 
HOF, . 

Insert the shorter hexagonal emu he the allfmwpfnch info the 
HEXAGONAL, HOI.F OK the S L 'TSCPFW and then turn the: sf.tscrf.w 

clockwise: until tight 

OK 

TIGHTEN THE. OTHE.P setscrew 
ok 

INSTALL HFLT 

How? 

CONNECT HFLT TO f 'OTOR PULLEY AND CONNECT RELT TO pilMpplJLLEY 
HOW? 

POSITION PELT ON MOTOppiJLLKY AND POSITION BEILT ON puwppl.1LLEY 
HOW? 

PI ACE RELT IN GROOVE OF' MOTORPULLEY 
OK 

PLACE BFLT IN GPOPVE OF PUVPPULLEY 
HOW? 

IF YOU HAVE TROUBLE GETTING THE: HFLT AROUND THE PlMpPIILLEY TRY 
MOVING THF MOTOR WHICH MAY MEAN LOOSENING THE FASTENINGS TO ALLOW 
IT TO SI, IDE SI.DF WAYS 
OK 

ATTACH PELT TO MOTORPULLEY AND ATTACH BELT T.l PUM.PPIJLLEY 
HOW? 25 
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MAKK bFI T VFRY TIGHT BY ADJUSTING fHF. POSITION OF THF. UlTOR 

































c. 


Urnanization of the Feoort 


mentioned, we have ornanlzed work In the proiert un^pr 
four headings: nrohi*m-solvlna, natural l.anauaa«, vision, and 
systeri irteorafion. Each of these research areas entails two types 
of work. First there Is some "immediate" work that is now helnn 
incorporated into the demonstration system. second there is some 
lonaer term research that win have an imoact on future systems. 

Pather thar oroanizino the renort into separate sections 
correspond!no to the tour research topics, w e have decided to 
describe in section n the work in each cateoorv that underlies the 
AdtII 1075 system. This will dive the reader an idea of how the 
Present system works and an Un ;erStanilno of its deficiencies. Then 
in Section ill we shall describe the work whose impact will ne felt 
in later systems. 

In addition to the front-line research topics, the rrolect 
needs the help of several important supporting tasks. These are 
maintenance of and improvements to the QMSP l,anquaq e , maintenance of 
and improvements to the SRI Artificial IntelHoence Center computing 
system, and development of certain hardware itpms such as the 
scanning laser ranoe finder and interface equipment. These 
supnortlnq tasks wm pe described in Section TV. 

Section V lists the Publications and presentations of the cue 
project staff during this reportino nerlod. 
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COMPONENTS OF THE APRIL 1975 SYSTEM 


A. Introduction 

our 1^75 systetr has already inteqrated several abilities. it 
can qenerate and execute plans for assembly/dlsassembly at several 
levels of detail. It can answer queries from the apprentice about 
the status of the equipment. It can point at parts of the compressor 
and can name parts pointed to by the apprentice. It nas a 
rudimentary ability for two-way communication using speech. It is 
the purpose of this section of the report to describe lust what 
underlies each of these abilities. 

In Section b we shall discuss our work in generating and 
executing hierarchical plans. This w 0 rk has resulted In the 
development of a concept called the "procedural net for representing 
hierarchical plans." The basic work on the procedural net has been 
done by Earl Sacerdotl as part of his doctoral research. In Section 
C we shall describe the polntlnq abilities of tne system. Underlying 
pointing Is a "geometric model" of the compressor, work on this 
model and on pointing is largely the work of Jerry Aqin and Georgia 
Sutherland, with some Important early contributions by David Nltzan. 


A 





\ 


A 



In Section D, a modeling package developed by Richard Flkes 
Is described. This package is used for raaklm deductions on the 
model In order to answer queries from the apprentice and from other 





















carts o£ the system. The package also provides a means for updatlnq 
models of the world during Pianninq, The current system uses some 
features of this package, and we have plans to use additional 
features soon. 

Our simple system for voice I/O is described in Section F. 
It consists of a speech synthesizer and a word recoqnlzer--both 
commercially available. 

Finally, in Section F we discuss some of the problems of 
system Integration. Georoia Sutherland has been responsible for this 
task, and it necessarily has Involved her in each of the others also. 

B. Reasoning About Assembly/Disassembly Actions 

1 . Introduction 

The assembly and disassembly of equipment Is a 
subtask ol virtually an workstation tasks. For example, many 
troubleshooting jobs and almost all repair jobs require some amount 
of disassembly and reassembly of the machine. So a major aspect of 
the consultation task is to model the actions entailed In assembly 
and disassembly, to compose sequences of these actions to accomplish 
specific tasks, and to model and monitor the execution of these 
sequences by a apprentice. 

Since the execution of a task is done under a mixed 



















initiative regimen, allowing the apprentice to andertakp subtisks in 
the manner he chooses, we cannot simpi/ otiiid a valid fixed sequence 
actions to accomplish a task. Father, we must inclucfp in our 
model the apprentice's implicit freedom to order the individual steps 
h i rn s e 1 f . 

In addition to develooino a metnodology for puildinq 
and executing hierarr:hica 11 y organized plans of action that reflect 
this freedom ot ordeilna, we are ieveloDing methodologies for 
defining a hierarchy of relations to descrine states of Partial 
assembly, and for deflninq a nlerarchy of actions to alter those 
states . 

In this section we shall first Jiscuss the integrated 
problem solving and execution monitoring system developed during the 
last year. Then we snail present the relations and action models that 
are being used to describe electromechanical equipment, with Portions 
ot the semantic model oi the air compressor included as an example. 
Finally, we shall descrloe how the system develops the dialog 
presented in Section I, 

2. An Integrated Problem Solving and Execution 

Monitoring System 

The N UAH (Pets Of Action Hierarchies) program 
combines a general rurpose problem solver with an execution monitor 
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that Is specifically designed for man-machine cooperation. In this 
section we shall present a simplified explanation of the procedural 
net, uGAH' s representation for actions and plans, of SOUP, the 
lanquade tor qivlna the system tas<-speciflc Knowledge# of the 
planning algorithm, and of tne execution alaorithm, A complete 
discussion of the system will appear elsewhere 1171 . 

NUAh is Implemented in ULISP 1181# and runs as 
compiled code on a PDP-10 computer under tne TENEX tlme-sharina 
system, 

a. The Procedural -Met 

The system's plans are built up in a data 
structure called the procedural net, w;,ich has characteristics of 
ooth procedural and declarative representations. 

Basically, the procedural net is a semantic 
network of nodes, each of which contains procedural information, 
declarative information, and pointers to other nodes. Each node 
represents a particular action at some level of detail. The nodes 
are linked to form hierarchical descriptions of operations, and to 
form Plans of action. 

Nodes at each level of the hierarchy are 
linked in a partially ordered time sequence by predecessor and 















sjccessor links. Each such sequence represents a plan at a particular 
level of detail. 

In the current implementation# the nodes are 
of six types: GOAL nodes represent a qoal to oe achieved; PhA'.n'OM 
nodes represent goals that are expected to be true when they are 
encountered; split nodes have a single predecessor and multiple 
successors, and represent a forking of tne partial ordering; JOIN 
nodes have multiple predecessors and a sinqle successor# and 
represent a rejoining of subPlans within the Partial ordering; Build 
nodes represent an action tnat builds uo a class of objects; BHEAK 
..odes represent an action that is iteratively applied to the elements 
of a class of objects. 

Each node points to a body of code. The 
action that the node represents can be simulated by evaluatina tne 
body. The evaluation will cause new nodes# representing more 
detailed actions# to be added to the net. It will also update a 
hypothesized world model to reflect the effects of the more detailed 
actions. 

Associated with each node is an add list and 
a delete list. These lists are computed when the node is created. 
They contain symbolic expressions representing the changes to the 
world model caused by the action that the node represents. 

Thus# the nodes of tne procedural net contain 
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two very different representations of an action. The add and delete 
lists provide a declarative representation of actions that is quite 
similar to that of STRIPS (lyj, The body of code provides a 
procedural representation similar to tnat of the new a.I. lanquages 
[201, The declarative representatinn is used to model the action at 
the node's own level of detail. The procedural representation is 
used for generating more detailed subactions at levels of greater 
detail. 


Figure 4 shows the graphic notation used here 
to display a node of a procedural net. 


Predecessor(s) \ 


V 



> 

-j- Successor(s) 


Node types are designated as follows: 


Description of action 

^ Description of action ^ 

S 


D 

GOAL 

PHANTOM 

SPLIT 

JOIN 


TA-740522-11 


FIGURE 4 GRAPHIC REPRESENTATION OF A NODE 


As an example/ let us examine a procedural 
net representing a hierarchy of Plans to paint a celling and paint a 
stepladder. The plan can he represented, in an abstract way, as a 
single node as shown in Figure 5(a). In more detail/ tne plan is a 


conjunction/ and might oe represented as in Figure 5(h) 


The more 




































detailed subplans to achieve these two goals might be "get paint 
ladder, then apply Paint to ceiling," and "get Paint, then 
paint to ladder," as depicted in Figure 5(c), 


LEVEL 1 


Paint the ceiling and paint the ladder 
(a) 


LEVEL 2 



(b) 

TA-740522-12 


LEVEL 3 



LEVEL 3 (c) 

(After Criticism 



(d) 


TA-740522-13 


, get 
apply 


FIGURE 5 PROCEDURAL NET FOR PAINTING 

































The pictorial representation used here 
suppresses much of the information associated with each node. The 
add and deiece lists, tor instance, are not indicated in the 
diaarams. They are not hard to inter, however, for example, "get 
ladder" win cause "has ladder" to aided to the wcna model, and 
"apply paint to ceiling" might delete "has paint" from the world 
model. 


Precondition-subgoal relationships are 
inferred by the system from Pointers that li dicate which nodes 
represent detailed expansions of other nodes. These pointers are 
also omitted in the pictorial representation. The system assumes that 
every action but the last in such an expansion is a precondition for 
the last action. 


TasK-Specific Knowledge 


Knowledge about the taslc domain is given to 
the system in procedural form, written in the SOUP (Semantics Of 
User's Problem) language, SOUP is an extension of ULISP 118) that is 
interpreted in an unusual fashion. The process of planning 
transforms this procedural Knowledge into the hybrid procedural net 
form, which contains botn procedural and declarative information, and 
whicn represents a hierarchy of solutions to the particular problem 
at hand. 
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we wm tirst present the statements of suuP 
tnat have been added to QL1SP. (We will call them P-statements.) Then 
we win describe how the SOUP code is interpreted. Specific examples 
of SOUP code are presented in Section Il.s.i.o. 

P"Statements that refer to actions are; 

PGOAL - 

A PGOAL statement is of the form: 
(PGOAl query pattern APPLY team). 

Its meaning is similar to the QLISP 
GOAL. it has an additional argument* 
tie query, that specifies a verbal 
request for the ooal to be ahieved. 
Evaluation of a PGOAL results in the 
insertion of a new node in the 
procedural net. It a true instance 
of the goal pattern is found in the 
world model, a PHANTOM node is created 
If no true instance is found, a true 
GOAL node is created. 































PBUILO 


A Pi'JILD statement is of the form: 
CPBUIlD class-name query (ITERATE.,,)), 
It specifies an action that *111 
build up a class of objects. The 
query is a verbal request to build up 
the class. The iterate statement 
contains an arbitrary body of code 
that specifies the subactions entailed 
in processinq one element of the class. 
Evaluation of a PBUILD statement 
results in the creation of a BUILD 
node in the procedural net. 


PBHEAK - 

PBREAK has the same syntax as PBUILD. 

It specifies an action that iterates 
through a pre-existing class of oblects. 
Evaluation of a PBREAK statement results 
in the creation of a BREAK node in 
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the net 


























A PAND statement Is of the form: 


(PAnD eyp-i exo-2 ... pxp-n). 

PANO specifies a collection of 

expressions that may be evaluated 
independently. Execution of a PAM) 
statement results in a n-way branching 
in the procedural net. 

P-statements that refer specifically to the 

world model are: 


PIS - 

A PIS statement is of the form; (PIS 
exp). It searches tor an instance 
of the exp that is true in the current 
world model. If none is found, it 
causes a failure condition. 

PASSEPT - A PASSEHT statement is of tne 
form; (PASSEPT exp). 

It maxes exp be true in the current 
world model and places exp on the add 
list of the current node in the 
procedural net, 
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POENY 


POENY is similar to PASSERT. It 
makes exp be false and adds exp to the 
delete list of the current node. 

When a P-statement that refers to an action 
Ci.e./ PGOAL, PbUlLD, PRRfcAK, or PAND) is evaluated, it does not 
cause an arbitrarily deep computation, as would most PLANNER-like 
languages 120J. Rather, the action is mocked (we mean simulated* 
but we're already using that term for another operation) and the 
world model is updated as if the deep computation had been done and 
the action were accomplished in full detail. The information 
necessary to continue the computation to further depths is stored as 
the body of code associated with the new nodes that are cr.ated in 
the procedural net. For example, when a PGOAL statement is evaluated 
(thus mocking the action that achieves the goal), the team of 
functions associated with the statement is placed as the body of the 
new node representing that goal. When a PBUILD or PBREAK statement 
is evaluated, the ITERATE clause is stored as the new node's body. 
This type of evaluation results In the creation of nierarchles of 
Plans of increasing detail. This scheme thus extends the ability to 
do hierarchical planning as was done by ABSTRIPS [21] from a 
syntactically oriented declarative representation to SOUP's 
semantically oriented procedural representation. 

c. Tne Planning Algorithm 

Initially* NOAH is given a goal to achieve. 
























.JOAH first builds a procedural net that consists of a sinqle goal 
node to achieve the given qoal. This node has i list of all relevant 
SOUP functions as its body, and it represents tne Plan to achieve the 
goal at a very high level of abstraction. Tnis one-step plan may 
tnen be expanded by the Planning algorithm. 

The planning alqoritnm of the NOAH system is 
simple. Its input is a procedural net. it expands the most detailed 
Plan In the net by simulating each node of the Plan in turn. In 
addition to building a detailed model of the effects of each action 
in the plan, tne simulation of each node will produce child nodes. 
Thus by simulating the Plan, a new, more detailed Plan wm be 
created. 


The individual subplan for each node will be 
correct, but there is as yet no guarantee that the new plan, taken as 
a whole, will be cor r ect. There may oe interactions between the new, 
detailed steps that render the overall plan invalid. For example, 
the individual expansions entailed in qeneratinq the plan in Figure 
5(c) from that in Figure 5(b) are correct, yet tne overall plan Is 
Invalid, since it permits Painting the ladcer before paintinq the 
celling. 


Before tne new detailed plan is presumed to 
work, the planning system must take an overall look at it to ensure 
that the local expansions make global sense. This global examination 























is provided by a group of critics. These critics serve a purpose 
somewhat similar to that of the critics of Sussman's HACKtR 122 \, 
except that for NOAH they are constructive critics# desianed to add 

constraints to as yet unconstra i ned Plans, wnereas for HACKER they 

were destructive critics whose purpose was to reject incorrect 
assumptions reflected in the plans. 

For example, a constructive critic win alter 
the Painting Plan in Figure 5(c) to ensure that tne endanqered 
suojoal, painting the ceilinq, is achieved before the step that 
endanqers it# namely painting the ladder. After this critic has 

altered the Plan# it will aPPear as in Fiqure 5(d). Not* that 

planners that use a linear representation of plans cannot solve such 
problems without extensive use of bacxtracking# or sophistlcated plan 
optimization. 

The algorithm for the Planning process# then, 

is as follows: 

(1) Simulate the most detailed plan in the 
procedural net. This win have the effect 
of producing a new, more detailed plan. 

(2) Criticize the new olan# performing anv 
necessary reordering o.r elimination of 
redundant operations. 















Clearly, tnls aiaorlthm is an 
oversimplification, hut for the purposes of this report we may 
imagine that the Planning process continues until no new details d re 
uncovered. (In fact, for the complete problem solvjno and execution 
monitoring system, a local decision must be made at every node about 
whether it should be expanded.! 

4 detailed example of the generation of a 
plan for assembling a disassembled air compressor is presented in 
Section 11.H.4. 

A recent paper [2 ^ J discusses in more detail 
the problem solving algorithms, tne constructive critics, and a 
comparison of this approach witn other recent work. 

<1. The Kxecution Algorithm 

The output of the planning process is a 
procedural net, which was developed as a hierarchy of partially 
linearized plans, figure 6(a) suggests the planner's viewpoint of the 
procedural net. The same procedural net is also the input to the 
execution portion of the system. The execution algorithm views the 
net differently, however. It sees the procedural net as a collection 
of action hierarchies, a s suggested by Figure 6(b). A n action 




























































hierarchy, consisting of a node representing an action, toqether with 
its children nodes representing subactions, toqether with tno ir 
descendant nodes, will be termed a wedge. Tne execution monitor 
views the procedural net as a sinale wedqe, to whicn it applies the 
following algorithm: 

(1) Ask the apprentice to accomplish tne 
action represented by the node at the top 
of the wedge. This is done by saying the 
node's query to hin. 

(2) It he resoonds Positively, assume the 
current wedge has been accomplished, and 
so assume the current wedge has been 
successfuly executed. 

(i) If he responds negatively, assume he 

neeos a more detailed breakdown of tne 
action, and so execute in turn all the 
subwedqes headed by children of the 
current wedge. 

Actually, the algorithm is more complicated 

than this. The apprentice may make a wider variety of responses. To 
each query, his possible responses are: 
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Affirmative responses 


Yes, OK# Yup# Check... 


This type of response Indicates tnat 
tne apprentice understands the 
instruction and is able to do it. 

In fact# in the current implementation# 
the CRC proqram assumes that the task 
has been completed. This type of 
response slqnais the execution 
alaoritnm to move on to the succeeding 
wedge. 

Negative responses - No# How, Can't... 

This type of response indicates that 
the apprentice needs help before he 
can perforin the indicated action. 

This signals the procedural net 
program to move to the first child 
node. 

Repeat responses - what# Huh... 

This type of response indicates that 
the apprentice did not hear the 
instruction or was not sure what was 
said. Please repeat. The program 


merely repeats the query 
























Motivation response - wny 


The apprentice wants to Know why a 
certain task needs to De done. The 
current response to this question is 
for the procedural net program to list 
tne tasks that remain to be done at 
that level. These correspond to the 
queries of successive nodes that have 
the same parent as the current node. 

If the apprentice still wants to know 
why, the program then repeats the query 
associated with the parent noae. This 
process may he repeated until there are 
no more parent nodes. 

Escape responses - break, Pause, Graphics... 

These responses are temporary expedients 
to allow interaction with other portions 
of the CBC system, such as the graphics 
package. (See the description of the 
CBC fcxpert System in Section II.F below). 
This allows tne apprentice the opportunity 
to ask questions like "where*" "what is 
tnis#" and the- like. 

When the top wedge of tne procedural net has 
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been successfully executed, the Execution Pnase terminates with 
"Thanic You". 


The current system's ability to monitor the 
execution of a 'ask: is limited, and its ability tc respond 


intelligently to unexpected failures is nil. During the coming year 
we plan to focus considerably more energy on the execution monitorinq 
aspect of the assembly/disassembly tasK. 


The Semantics of Assembly and Disassembly Actions 


Relations 


In order to model the processes of assembling 
and disassembling equipment, we have defined a set of entitles and 
relations that are used to describe a device in arbitrary stages of 
assembly and disassembly. The definitions being used in the current 


system are given in this section along with examples that relate to 
the air compressor. These definitions form the basis for the SOUP 


functions that produce a procedural net of Plans for assembly and 
disassembly. The SOUP functions will be described in Section 
11. B. 3. b. 


Connections 


A "connection" is defined between any 
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two components that are fastened to eacn other onen a device is tullv 
assembled. The elements of a connection are an ordered pair of 
components, where the orderinq implies that tne first component is 
fastened to tne second component, for example, we have (CONNECTION 
pump PLATFORM) ratner tna n (CONNECriUrt PLATFORM PUMP), since one 
would say "Connect the pump to the olatform" rather than "Connect the 
Platform to the pump". In the cases where no intuitive or actual 
assymetry exists between the components, we determine whether one of 
the components supports the other in the fully assembled device. If 
so, the supportee is the first element of the Pair. In the 
remaining cases, where neither criterion applies, an arbitrary 
ordering of the components is chosen. 

The connection relations are used for 
defining the canonical locations of components for the fully 
assembled device, rather tnan for specifying actual locations of 
components at any Particular time. Thus, connection relations remain 
unchanged during assembly and disassembly. 

ii) Fastenings 

Connections usually represent 
components that are fastened toaether ny bolts*, screws, machine 
screws, or setscrews. The objects that fasten components together 
are referred to as "fastenings". A bolt with its nut and washers is a 
standard type of fastening. Examples of fastenings Include each of 
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the tour nut-washer-tolt assemblies that connect the pump to the 


Platform and each ot the ten sheet metal screws that fasten the belt 


housinq cover to the frame. 


The association between a fastening 


and the components it fastens is represented by storing with each 


connection tne fasteninas that hold it together. Table 1 lists some 


of the connections and fastenings that are currently used by the CBC 


system 


Table l: CONNECTION Expressions for the Air Compressor 


(CONNECTION PumP Platform) 


FASTENFNsdOLTON 


FASTENING=(_BciLT Pumpbase Platform -WASHER .NUT) 


LENGTHS 1.25 


DIAMETEHs.3125 


N11M b E R = 4 


(CONNECTION Motor Platform) 


F ASTENFNsBOLTON 


FASTENING=(.BOLT M 0 torbas e Platform —WASHER .NUT) 


LENGTHS 1 


DIAMETER s.25 


N U M b E R = 4 
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(C0NNLC1I0N Pumppulley pump) 

P A5TENFN = *vHKLL0N 

f ASTENING=(_SP'.TiSCHfc. 1 « pumppulley PumDShatt) 
LPINGTPis,2b 

DIAMETtHs.25 

NUMbFRs? 

(CUNNpxtion Motornuiiey Motor) 

P ASTENFNsWHP.P'.LUN 

P AS f LNI MG= (_SETSC RE w Motorpuiiey Motorsnatt) 
LF.NGTHs,2b 
DIAMfc.TEHs.25 
MllMHFRsl 

(CONNECTION Belt pumppulley) 

P ASTENFNsRfc'.LTON 

(CONNECTION belt Motorpulley) 

P ASTfc NFNsHEl.TON 

(CONNECTION PumPbracr Pump) 

P ASTENFNshOLTON 

FASTEnInGs(_BuLT Pumpbrace Pumptoo) 

LENGTHS .75 
DIAMF.TERs.3125 

1 U M h P' H = 1 
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(CONNECTION Pumpbrace BeLthousingframe) 

F ASTENF N = SCKEKllN 

F ASTENINGs (.SCREW Pumobrace Del thouiinQf rarr>e ) 

LENGTHS .5 

DIAMETERS.25 
NUMBFRs2 

(CONNECTION Belthousinacover Be 1thousingtrame) 
FASTENF'NsSCRENON 

Fasten lNGs(.SCRE ^i Be 1 1housingcover Beithousinaframe) 
LENGTH=.5 
DI AME'i ERs , 2b 
NUMBERS 10 

(CONNECTION Aftercooler Pump) 

FASTENFNsSCHEWInTO 
FEMALEspumP 
MALF=Aftercooler 

(CONNECTION Attercooler Attercoo1ereloow) 

FASTENFNsSCREwlNTO 

FEMALEsAftercooler 
MALE=Aft£.rcoolerelbow 













Ill) Chunks 


T*o components x and Y nave a 
connection path defined o e t w e e n them it there exists some sequence ot 
connections betw<en components X-ci, C1-C2, ..., Ch-Y. A "Chunk" is 
a collection ot components that have connection paths defined amon^ 
them and are all positioned with resoect to each other; i.e., a 
positioned supasrembly. we define tne "loose ends" of a chunk to be 
that collect.ion ot connection relations between mempers of the chunk 
and nonmemhers of the chunk. For example, an isolated single 
component can be considered a chunk and all the component's 
connections would be considered loose ends. If a chunk is positioned 
with respect to some other chunk, then the loose ends set of the 
newly formed (larger) cnunk consists of those connections that were 
elements ot exactly one of the loose ends sets of th<_ two old chunks 
(since connections that were loose ends of both the old chunks are 
now connections between members of the new chunk.) 


Fastened/Unfastened 


!f X is a fastening, then the 
relation (FASTENED X) is true whenever X nas been inserted in its 
proper place and tightened. For example, a fastened nut and Dolt 
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assembly means that the bolt# nut# and all associated washers are 
firmly In place. Similarly# (UNFASTENED X) is true when the fastenlnu 
X Is disassembled and removed frcm its proper position. 


v) Positioned/Extracted 

If components X and Y have a 
connection defined oetween them# then tne relation (POSITIONED X Y) 
Is true whenever X and y are in the same chunk. Similarly, if 
components X and Y have a connection defined Detween them, then the 
relation (EXTRACTED X Y) is true whenever X a n d Y are in different 
cnunks and X and Y do not restrict each other's movement in any 
meanindfui way. 


Chunk descriptions and the relations 
POSITIONED and EXTRACTED are redundant# since being a loose end 
implies EXTRACTED and not being a loose end implies POSITIONED. 

It is convenient to define the 
relational form (POSITIONED X) to be true for a component X if and 
only it (POSITIONED X Y) is true for each Y such that (CONNECTION x 
Y) is defined. Similarly# (EXTRACTED X) is defined to be true for a 
component X it and only if (EXTRACTED X Y) is true for each Y such 
that (CONNECTION X Y) is defined. 

vi) Attached/Detached 

If (CONNECTION X Y) is defined# then 
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the relation (ATTACHED X Y) is true wrenever (FASTENED Z) is true for 
an fasteninas Z associated witn the X-Y connection. (Note that 
ATTACHED and FASTENED can only oe tru? it positioned is also true.) 
Similarly, it (.CONNECTION x Y) is defined, then the relation 
(DETACHED x i ) is t rue whenever (UNFASTENED z) is true tor all 
tastenlnqs Z associated with the X-Y connection. 

We also define tne relational torm 
(ATTACHED X) to be true tor a component X it and only it (ATTACHED X 
Y) is true tor each Y such that (CONNECTION x Y) is defined. 
Similarly, (DETACHED X) is aetined to be true for a component X if 
and only If (DETACHED X Y) is true for each Y such that (CONNECTION X 
Y) is defined. 

vli) Connected/Disconnected 

For any components X and Y, the 
relation (CUNMECTED X i ) is true whenever (ATTACHED X Y) is true, and 
tne relation (DISCONNECTED X Y) is true wnenever (EXTRACTED X i ) is 
true. (Note that ATTACHED implies POSITIONED and EXTRACTED implies 
DETACHED.) 

we also define the relational form 
(CONNECTED X) to re true for a component X if and only if (CONNECTED 
X Y) is true for each Y such that (CONNECTION x t ) is defined. 
Similarly, (DISCONNECTED is defined to be true for a component X 
















If and only it (lISCO unkc TKD X Y) is true for each Y such that 
(COhhhCTUlN X K) is defined. 

viii) 1 nstalied/Removed 

Kor any two comoonents X and Y ot the 
sane device, tne relation (I .ySTaijLED X Y) is true whenever X and Y 
are in the same churk ano when (COnue'CTtu X Z) is true tor each Z (in 
the chunk with A) tor wnich (CUVNtCTlUN X Z) is defined. Also, If X 
and Y are components of the sane device, then the relation (RFMOVKD X 
l) is true whenever x and Y arc in seoarate cnunks and X and Y no not 
restrict each other's movement in any meaningful way. 

we also define tne relational torm 
( INSTALLFU X) to he true for a component X it and orly it (iNSTALLtl) 
X Y) is true for each i such that (CUMhECTlOi X Y) is defined. 
Similarly, iHLiwoveo X) is defined to oe true for a component X if and 
only if (KHluVfcD < Y) is true for eacn Y such that (CON'JfcCTIUN X Y) 
is defined. 


lv) Assemoied/LHsassemhied 

For any device X, tne relation 
(ASSi> ! hl.fcn x) is true whenever ( 1 .«STALbfcu Y) Is true for each 

component Y of x. similarly* tor any device X, the relation 

(OfSASShhttLtr X) is true wnenever (kKMHvFu Y) Is true tor each 

component Y of X. 






Onhand 



x ) 


It X is a component of a fastening or 
is a tool, then the relation (Onhand <object-type of X> X) is true 
whenever A is i n use by or immediately available to the technician. 
This relation is typically used for objects such as screws, colts, 
nuts, washers, setscrews, wrenches, or screwdrivers, 

xi) Startedin/Taxenout 



It X is a fasteninq, then (STARTEDlh 
X) is true whenever the components of X have been positioned with 
respect to the connection th<.t they fasten. Similarly, if X is a 
fastening, then ITAKtMOUT X) is true whenever the components of X 
have been removed from tne connection that they fasten. 

xI i) Tightened/Loosened 


J 


i 


k *' 

l U 

( 


Nr 



If X is a fasteninq, then (TIGHTENED 
X) is true whenever the components of X have been tightened. For 
example, a bolt-washer-nut fastening is tightened by turning the nut 
onto the bolt until tight. Similarly, (LOOSENED X) is true whenever 
the components of a fastening are loosened. The relations 
STARTED1N/TAKENOUT and TIGHTENED/LOOSENED for fastenings are 
analogous to the relations PQSIT 1DNto/tXTRACTED and 
CONNLCTFD/DISCUNNECTFO for components. 
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xtii) 


Spindled 



SPlwDLLD is a relation tnat applies 
to a fastenina and a list of components. Tne SPINDLtu relation* if 
true* implies that the objects are fastened together in the order 
listed in the relation. An example is (SPJNDLtD boltl Pump Platform 
rfasherl Nutl), This type of relation is used to keep track of 
associations between particular bolts, nuts, washers, and the like. 

xiv) Applied 

The relation (APPLIED <tool-type> 
<fastening-name>) is true whenever a tool of the specified type is 
beina used by the apprentice to tighten or loosen the specified 
fastenina. 






b. Describing Actions for Assembly and 

Disas semb ly 

The relations described in the preceeding 
section imply a hierarchy of actions to be used in forming plans 
about assembly and disassembly of equipment. Accordlnaly, SOUP 

functions nave been written to achieve all the relations, referring 
explicitly to tools and fastenings when appropriate. 
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Flaures 7 and 8 snow the relations/actions 
hierarchy lor assembly and disassembly planning. The upper case 
names refer to relations; the actions, or SOUP functions 
correspond1na to those relations/ are lower case names. The vertical 
lines connecting the blocks in tne figures indicate different levels 
of the hierarchy, and imply function calls. wnen a SOUP function 
calls more than one lower level SOUP function, tnis is indicated by a 
horizontal line. Some of the assembly actions are described below. 

ASSEMBLE (ULAMBDA (ASSEMBLED -OBJ) 

(APPLY (FUNCTION PANP) 

(MAPCAK (GETp $OBJ (OUUTL COMPONENTS)) 

(FUNCTION (LAMBDA (X) 

( f ( L 'GOAL (INSTALL (« X)) 

(INSTALLED (P X)) 

APPL t (INSrALLOWJ)))J 

A list of COMPONENTS has been Previously set up. To assemble the 
object, set up a goal for installing each component separately, if 
any component is already Installed, this information win be already 
stored in the world model, pGOAi. will discover this tact, a PHANTOM 
hode will be Inserted in the net, and the apprentice will never be 
instructed to Install a component unnecessarily. 
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FIGURE 8 FLOW CHART FOR SOUP FUNCTIONS IN THE DISASSEMBLY DOMAIN 
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1NSTALL0BJ (OLAMBDA (INSTALLED -UBJ ) 

(UPRUG (-0BJ2) 

(APPLY (FUNCTION PAimD) 

(MAPCAH (CUP (INSTANCES 

(CONNECTION $UBJ -U8J2))) 

(FUNCTIUN (LAMBDA (X) 

(5ET0 X (CADDH X)) 

(t(PGOAL (CONNECT $OBJ TO (@ X)) 

(CONNECTED SOBJ (<<* X)) 

APPLY (CONNECTOBJ ) ) ) ] 

Each defined connection relation Is explicitly stored In the world 
model. The GLISP function, INSTANCES, will retrieve all those 
connections that Involve $GBJ. Then a qoal Is set up to connect 
SOBJ to an other components. Notice that the P-fUnction* PANU, is 
used to make a conjunction of all the connections! the procedural net 
critiques will put the conjuncts In the best order for execution. 

CONNECTORJ (OLAMBDA (CONNECTED - OBJ 1 -OBJ2) 

(PGOAL (PUSJ TI ON $UBJl UN $UBJ2) 

(POSITIONED $013 J1 $0BJ2) 

APPLY (POSITIONOBJ) ) 

(PGOAL (ATTACH $0BJ1 TO $0BJ2) 

(ATTACHED $0BJ1 $0BJ2) 

APPLY ((?(QG£T (CONNECTION $GbJl $0BJ2) FASTENFN))))J 


















and 




In order to connect two objects, they most first be positioned, 
then attached to each other. The type of attachment is implied by 
the fasteninq function property, F ASTENFN , stored on each connection 
relation, 

POS1 T1 ONCCiJ (QLAMdDA (POSITIONED *-OBJl *-UbJ2) 

(OPROG (-MOBJ - FOBJ) 

(SELEC Tu (UGET (CONNECTION $OBJl $UBJ2) FASTENFN) 

(wHEEliON (PGOAL (PLACE THE SUBJl ON THF SUBJ2 

SHAFT IN SUCH A WAV THAT THE 
FLAT SIDE OF THE SHAFT IS ALIGNED 
WITH THE FLAT PART INSIDE THE 
CENTER HOLE OF THE $0BJ1 AND THEN 
PUSH THE SOBJ1 ONTO THE SOBJ2 
SHAFT AS FAR AS YOU CAN) 

(POSITIONED SOHJ1 $OBJ2) 

APPLY NIL)) 

(HELTON (PGOAL (POSITION $OHJl Jn GROOVE OF $OhJ2) 

(POSITIONED SOPJJ SOBJ2) 

APPLY (I'XPLA INHELTPOSIT IOh ) ) ) 

(SCPEwINTO (OGET (CONNECTION $0HJ1 $OBJ2) 

FEMALE «- FOBJ MALE <- MOB J ) 

(PGOAL (PLACE THE END OF- THE $MOBJ 


NEXT TO THE HOLt UT THE $FOBJ) 
(POSITIONED $UBJ1 $OBJ2) 

APPLY NIL)) 





















(PGUAL (PLACE THE $OBJl AND THE S0BJ2 NEXT TO EACH 
OTHER IN SUCH A WAY THAT HOLES IN THE SOBJ1 
ARE MATCHED UP WIH HOLES IN THE $(>BJ2) 

APPLY ML)))] 

This is a general positioning function that gives instructions about 
four different types ot Positionings: Wheels# belts, components that 
screw into other components* and, finally, components tnat are 
connected with fastenings such as bolts, or screws. 

BOLTON (QLAMBOA (ATTACHED -UBJt -OBJ2) 

(PGOAL (USE BOLTS TO CONNECT $UbJl AND $OBJ2) 

(FASTENED BOLT ($OBJl SOBJ2)) 

APPLY (FASTENOBJ ) ) ] 

This is an example of an attachment function, applying to components 
that are to be bolted together. 

FASTENOBJ (ULAMBDA (FASTENED -FASTENING (-OBJ1 - OBJ2) ) 

(QPRUG (-OBJECTS - LENGTH -N -DIAMETER -C 
-COMPUTATION -VARIABLES 

(-FASTENINGS (MKATOK (CONCAT $FASTENING "S")))) 

(OGET (CONNECTION sOBJl S0BJ2) 

FASTENING -OBJECTS LENGTH -LENGTH 
NUMBER - N SHAPE -SHAPE DIAMETER - DIAMETER) 

(COND ((EQ $N 1)(MATCHQ -FASTENINGS P^ASTEN1NG) ) ) 






























(We have omitted a section of code nere that sets SVARlABLFS 
equal to the list of variables in SUBJECTS. These 
are the holts# washers# nuts# screws# etc.# which 
are not explicitly specified by name. The code 
constructs a query of the form "net two #3 holts and 
two #4 washers and two #3 nuts” to be used in the 
subsequent PBl'ILL) statement. This special query is 
saved as the value of SCONFUTATION.) 

(PBU1LD (CLASS --C) 

(GET ( ! (@ (C!)R (APPLY (FUNCTIUN APPEND) 

scumpu ta riuN j))j) 

(ITf-RATE $n $ VARIABLES 

(MAP2C SCOMPUTATION SVAR1ARLES 
(FUNCTION (LAMBDA (X Y) 

(PGOAL (GET ONE (I (a (CDL)R X)))) 

(ONUAJD (W (MKCONSTANT Y)) (fa Y)) 

APPLY (LUUKFOROBJ) 1YPE (fa (CDDR X))))) 
(QPUT (CONNECTION SUPJ1 $OBJ2) HARDWARE $C) 

(PBHEAK (CLASS -«-C ) 

(LOOSELY FASTEN THE $ N sFASTE'UNGS) 

(ITERATE $N SVAR IABLES 

(PGOAL (START IN UNE $FASTENING) 

(STARTEDIN $FASTENING (@(t $VARIABLES))) 
APPLY (STAR T lN)) 

(MAPC $VARIABLES (FUNCTION (LAMBDA (V) 

(SETQ V (LIST (QUOTE UNHAND) 

(MKCONSTANT V) V)) 

(PDENY (9 (t (0 V))))))))) 
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(PBHLAK (CLASS —'C ) 



(TIGHTEN THE $N SFASTEMNGS) 

(ITERATE $N $VARIABLES 

(PGOAL (TIGHTEN UNK $KASTEMNG ) 

(TIGHTENED $FASTENING (P( t $ VAR1ARLES))) 
APPLK (TIGHTENBOLT 

TIGHTENMACHINESCREW 
TIGHTENSCREw 
fIGHTENSETSCREW)) ) ) J 



In fastening objects together, first get the fastenings, then loosely 
fasten them, then tighten them. The initial QGET statement retrieves 
the properties of the fasteninas themselves (length, diameter, 
number, and so on). The PBUILD statement instructs the apprentice to 
physically get the tasteninqs. The two PBREAK statements tell the 
aoprentlce what to do with the fastenings to matte the connection. 



TIGHTEMbOLT (QLAMBDA (TIGHTENED BOLT (-.BOLT --0)) 

(QPRDG (*- WRENCH 1 - WRENCH2 - NUT ) 

(COND ((OR (NULL $0) 

(NUT (EQ (GETP (MATCHQ —NUT (FLAST $0)) (QUOTE TYPE)) 
(QUOTE NUT))))) 

(PGOAL (FIND A wRENCH TO FIT THE BOLT HEAD) 

(ONHANQ WRENCH -WRENCH2) 

APPLY (LOOKFOROBJ) TYPE FRENCH) 

(PDENY (UNHAND WRENCH $WRENCH2)) 









(PGM All (USE THF WRENCH TC TURN THE BOLT 
CLOCKWISE UNTIL FAIRLY TIGHT) 

(APPLIED SWRENCH2 $BOL l ) 

APPLY (USEWRENCH))) 

(T (PGUAL (FIND A WRENCH TO FIT [HE NUT ) 

(UNHAND WRENCH - WHENCH1) 

APPLY (LUUKFORObJ ) TYPE wRFNCH) 

(PDFNY (UNHAND WRENCH $WRENCH1)) 

(PGUAL (FIND A WRENCH TO FIT THE B(JL1’ HEAD) 

(UNHAND WRENCH *- WRENCH2 ) 
apply (LOOKFURUBJ) TYPE WRENCH) 

(PDEMY (ONHAwD wRENCH SWRENCH2)) 

(PGUAL (USE THE FIRST wRENCH TO HOLD THE NUT FIRMLY) 
(APPLIED SWRENCHl $NUT) 

APPLY (USEwRENCH)) 

f PGUAL (USE THE SECOND WRENCH TO TURN THFJ BOLT 
CLOCKWISE UNTIL FAIRLY TIGHT) 

(APPLIED $WkEwCH2 $B0LT) 

APPLY (USEWRENCH)) 

(PASSERT (ONHAND WRENCH $WRENCH1)))) 

(PASSERT (UNHAND WRENCH $WR£NCH2 ) ) ) 


























This is an example of a particular tiqhtenina function. 

(TIGHTENSCRFW, TIGHTLNMACHINESCREW, and TIGHTENSETSCREW are Similar.) 
The process requires the apprentice to get the tools and use them. 
While the tools are in use, they are declared to be "unavailable" by 
the PDENY statements. They are later made available again Dy 
PASSERT. 

It is important to notice that none of the 
SOUP-functions described in this section actually refer to an air 
compressor in any way. They refer to members of a COMPONENTS list 
and to a set of CONNECTION relations that are stored in the world 
model. Tools are explicitly referred to in the lower level 

functions, as are fastenings such as screws, nuts, and bolts. 

All the specific information about the air 

compressor is coded into a single initializing function that sets up 

the world model before any of the procedural net is built. The 
initialization function contains many expressions of the following 
f orm: 

(ASSERT (CONNECTION PUMP PLATFORM) 

FASTENFN BOLTON 

FASTENING (-BOLT PUMPBASE PLATFORM -WASHER -NUT) 

LENGTH 1.25 
DIAMETER .3125 
NUMBER 4) 

(Other examples were shown in Table 1 above.) 








Thus we expect that It would be relatively easy to 
convert the current planning and execution monitoring program to 
another example ot electromechanical equipment. 

4. Examples trom a Demonstration System 

a. A Detailed Irace of the Planning Algorithm 

The NfiAH system is invoked with a top-level 
goal as its argument. An initial procedural net is hunt, with the 
structure shown in Figure 9(a). Tne body ot the goal node is the 
value of a QLISP variaoie that is bound to a list of ail toP-leVel 
SOUP functions (in this case, ASSEMBLE and DISASSEMBLE). 

The planning algorithm is then applied to tne 
most detailed plan in the initial net. At this point, the most 
detailed plan consists ot the single GOAL node. Tne GOAL node is 
simulated, which means that the functions in its body, namely 
ASSEMbLE and DISASSEMBLE* are applied in turn to Its pattern 
(ASSEMBLED AIRCOMPBESSUR) until some function does not fail, (Ngtp 
that this is essentially equivalent to evaluating the QLISP statement 
(GOAL (ASSEMBLED AIRCOMPRESSUR) APPLY (ASSEMBLE DISASSEMBLE)),) 

The function ASSEMBLE was reproduced in 






















(b) 
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PROCEDURAL NET OF THE DEMONSTRATION SYSTEM 























Section 3 above. It evaluates the exoression (PANP (PGOAL (Install 
CCMP-i) (INSTALLED COMp-j) APPLY (INSTALLObJ)) ... (PGOAL (Install 
COMP-n)(INSTALLED COMP-n) APPLY (INSTALLObJ))), for components COMP-1 
throuqh COMP-n of the air compressor. Evaluation of the pand 
statement results in the evaluation of each of the PGOAL statements, 
^hen each PGOAL statement Is evaluated, the expression (INSTALLED 
COMP-i) is asserted in the current wjrii model, and a new node is 
placed In the procedural net. The new node will have (INSTALLED 
COM p -i) as * ts Pattern, and wm have (INSTALLOdJ) as its body. 

Because the PGOALs are Inside a PAND 
statement, the nodes that they qenerate in the procedural net are not 
linked linearly. Rather, they are linked In Parallel to special SPLIT 
and JOIN nodes, as depicted in Figure 9(b). rhls completes the 
simulation of this level of the Plan. 

At tnls level the critics do not yet have a 
sufficiently detailed model to analyze, ana so they propose no 
alterations to the plan. 

Next the new, more detailed plan is 
simulated. This mpans that each of Its nodes, In turn, win be 
simulated. First, the SPLIT node Is simulated, and this results In 
the creation of a new SPLIT node at a lower level of the procedural 
net. Then, each of the GOAL nodes is simulated, by applyina the 
function INSTALLObJ to its pattern. INSTALLOBJ con'ains a hand 
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statement (see Section 3 above). Each application of INSTALLOBJ 
represents the installation o'f a particular component. Evaluation of 
the PA ML statement in i:"?TALLOBJ wm result in the generation of a 
parallel qrc'p of GUAL nodes specifying all the connections to be 
nade to the given component in the fully assembled air compressor. 
Finally, the JOIN node is simulated# resulting in the creation of a 
new JOIN node. The new, most detailed level of the procedural net 
looks like Fiqure 9(c) at this point. Now# it preconditions had been 
defined 'or each connection, the critics would be able to modify this 
structuis to appear as in Figure 9(d). However, the preconditions 
have not as yet been encoded# and so the demonstration system relies 
on the intelligent initial orlering of the components list by the 
detiner of the domain. The critics do# however# clean up any 
superfluous SPLITS and JOINS, 

Expansion of the net proceeds in this manner, 
until a predetermined depth has been reached. In the future the 
depth of expansion win be determined by a model of the apprentice's 
capabilities. Further expansion can, of course, occur durinq 
execution of the plan if the apprentice requires more detailed 


Instructions 















GURE 9 PROCEDURAL NET OF THE DEMONSTRATION SYSTEM (Continued) 







































b. A Sample Hialoo 

A samDle dialog was aiven In Section I.B.3 
above. It exhibited an interaction Detween the CBC expert (FJ) and tne 
apprentice (A) and contained a few annotations and explanations of 
the interaction. This type of dialog is produced from the execution 
of a previously-constructed procedural net, although it is possible 
to have the net constructed "in real time" as the apprentice gives 
answers to the instructions of the expert. A given procedural net can 
produce many variations of dialog according to the level of 
experience of the apprentice. This makes it a Dowerful base on which 
to build the rest of tne system's capabilities, as will be described 
in succeeding sections of this report. 

C. A Geometric Model Used for Pointina in the C PC Prnle.t 

1. Introduction 

We have devised a method of computer modellnq of the 
shapes of mechanical parts and assemblies, and we have devised a 
number of aloorithms for manipulating the models. These have been 
Incorporated into a demonstration nroaram modeling the air 
compressor. The proaram constructs models of the compressor from 
symbolic descriptions, djsoiays an image in Perspective, and mav be 
used to identify or point to various parts of the compressor, by 
means of a television camera anl laser bolnter. 
























The computer representation of rorrplex assemblies Is 


built from primitive sol Ms (cubes* w<»daes, ani cylinders.) In this 
respect, the models are similar to those described bv I'raid f24l and 
others [2S, 2b], The principal character 1s11cs which we believe to 
be unique in this implementation are the use of symbolic "attachment 
points" to specify the relative positions of Parts with resoect to 
one another, and the retalntnq of symbolic and descriptive 
information alone with purelv oeometrlc information, 

fee use the nodels in the followlnq ways: An 
apprentice miay "point to" various parts of the compressor by toueninq 
that part with a wand with a small llqht bulb at Its tip. The 
computer responds by namino the part. Alternatively, the 
apprentice may ask the computer to aim the laser pointer at a named 
part. 


The system at present m a v be described as "qraPhics 
oriented", In the sense tnat It Infers appearances from geometric 
modPls» rather than Inferring aeometrv and structure from appearances 
as a "vision oriented" system would, fee are In a preliminary phase in 
the develonmert of the modellno capability. As we gain experience, 
the system will pp extended to provide a basis for scene 
understanding in the workshop domain, and to do geometric reasoning 
for Planning purposes. 
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The demonstration system consists of two modules for 
"forks" in TFNF.X parlance), one ooerating in a LISP environment and 
one operation under SAIL, The MSP Portion manages symbolic 
descriptions to determine the structural and snatlal relations amono 
parts. The SAIL portion keeps the specific neometric descriptions and 
absolute spatial Positions, and also contains algorithms for 
man 1pu latino visible outlines. The symbolIc/structura] data 
structures and algorithms are described in the next subsection, and 
the oeometrIc/spatlal in the one followina. 

?. Symbolic and Structural Models 

The basic unit of structure in our modeling system Is 
the "part". Descriptions of complex objects are built hierarchically 
from simpler objects. Thus a part may represent something as simple 
as a primitive huiidlna block, nr as complex as the entire 
workstation. Associated with each part are a number of properties 
that describe its structure and position, and define attachment 
points for use in assembling parts into hiuher-level assemblies. 

Two kinds of parts are used. "Model parts" make use 
of dummy parameters to specify their variat le dimensions, and are 
usualiv described in a canonical Position. "Actual carts" are conies 
of model parts that have a specific position and orientation in space 
and dummy parameters replaced by actual numbers. Model parts are 












crpareci by a prooram^pr or desianer in soeclfVinq a piece of 
eauipment; actual Darts arc created by the nrooram. in either case, 
all Information about a cart is carried on the property list of the 
LISP atom representino the part. 

The system is best described by example. Consider 
the description of the air compressor's DUmn. For the purposes of 
qeometric modelinn, the pump is modeled as two rectilinear mocks 
stacked as shown In Ploure in. The model Part pump contains on Its 
oropertv list the follow inn: 

structure 

[(CRANKCASE (BRICK 5.0 3.5 5.51) 

(PISTDM-CYI INPFR (HR[C K 3.1 3.1 5.0) (RFF CRANKCASE TOP))] 

This description says that the pump is the union of 
two simpler parts. The first item in each sublist is a symbolic name 
to be used within the context of the nump to refer to the subparts. 
The two supparts of the pump are assiqned the symbolic names 
CRANKCASF and PIST0N-CYL1 M PEP t The second item in each subllst 
describes how to build each subpart. The CRAN*CASF 1s be modeled 
ds a brick of dimensions 5.0 x 3.5 x 5.5 inches and the 
PISTON-CYLINDER as a brick of dimensions 3.1 x 3.1 x 5.0 inches. 

The remainder C f each subllst tells how the subparts 
are to be Positioned within the assembly. Since the crankcase 
description contains no explicit position, the location of the 
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FIGURE 10 GEOMETRIC MODEL OF PUMP 
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crankcase is the same as the location of the Dumo itself. The 
location information for the PISTON-CYLINDER states that It is to be 
placed at the top of t.he crankcase. To find the definition of "too"* 
it is n^cessarv tp examine the description of PRICK, The complete 
description of RRICK Is as follows: 

STFUCTUFF PRIMITIVE 
PARAMETERS (WIDTH DEPTH HFIGHT) 

ATTACHMENTPDTNTS 

I (BASE (MTPSP 0 0 (MINUS (FQUOTIEMT HEIGHT ?)))) 

(TOP (MTRSP 0 0 (FQUOTIENT HEIGHT ?))) 

(BACK (MTRSP 0 (FOtKlTTENT DEPTH ?) 0) (MROT "X" - 90.(1)))] 

The attribute PRIMITIVE under the property STRUCTURE 
implies that there are no subparts to a brick, (There exist routines 
In both the LISP and SAIL proarams for deallna aeometrleally and 
spatially with bricks and other primitive Darts.) The PARAMETERS list 
contains symbolic names for the brick's dimensions* which are 
Indicated in Fiqure 11. Finally* attachment Points are provided for 
olacinq other objects on or near an example of a brick. Since our 
description of the pump depended on the Tr)P of a brick, let us 
examine its attachment ooints in areater detail. 

An attachment point is a rule for flndino a specific 
point on the surface of a part from an arbitrary ooint on or within 
the part. A brick is defined qeometrically in the SAIL fork in a 
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FIGURE 11 BRICK. 


canonical Position and orientation entered on the nriom of 
coordinates, ’lowever, it Is more convenient io think of the location 
of a brick as the location of a Point on which the brick m a y he 
placed. The BASK attachment point allows us to perform this 
transformation automatically. When we wish to specify ihat a cube of 
? Inches is to be maced on a table lo Inches off the floor, the RASE 
attachment point tells us the base is located height/? (or 1 Inch! 
below the cube's center. (The function <TRSP represents a translation 
In x, y and z.) T hus the center is one inch above the tabletop, or .11 
Inches above the floor. Furthermore, the ton of th. e cube Is located 
one Inch above the cube's center, or 3? Inches above the floor. (When 
a part contains no explicit BASF, attachment point, as in the case of 
the pump, the null transformat ton is assumed. The RASE of the pump is 
the same as the point on which the C P A M K C A S F is to be placed.) 
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The attachment point BACK Is interesting in that it 
is a combination of two primitive transformations. To attach a cart 
to the back of a brick, must be first mover) PFPTH/? back from the 
brick's center, then rotated on decrees about the x-axls sc that its 
base may rest on the vertical hack face of the brick. (The function 
^ ROT generates a rotation about an axis.) 

A fair amount of processing is oecessarv to determine 
where a part is to he placed. Rut the main purpose in the use of 
attachment points is that once thev are defined, the programmer need 
not be aware of it. He need only specify the symbolic name of an 
attachment point and the computation Droceeds automatically. 

Parameters have not been well exploited in our 
description of the air compressor. Nevertheless we expect them to 
be Very useful when we model to a greater level of detail. For 
example, a slnole structural description may specify a generic RULT, 
with length, diameter, and head tvoe to he supplied in each specific 
instance, Washers, nuts, hearings, shafts, and other common parts 
may also be specified in this wav. 

Usina the mechanisms described above, model 
descriptions of the relevant objects In the workstation may he built 
up. Such a description is shown in Figures 12(ai through 12(h). 
















ROOM: STRUCTURE [ (TABLE (TABLE) 

(MTRSP 6.0 30.0 0)) 
(COMPRESSOR (COMPRESSORONBASE) 
(REF TABLE TOP)) ] 


TABLE: STRUCTURE [ (TABLE (CYLINDER .7 54.0) 

(MTRSP 0 0 27.6)) 
(TURNTABLE (CYLINDER .7 36.0) 
(REF TABLE TOP) 

(MTRSP 0 0 .95)) ] 

ATTACHMENTPOINTS [ (TOP (REF BASE TOP)) ] 



(a) ROOM 
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FIGURE 12 GEOMETRIC MODEL OF WORKSTATION 





























COMPRESSORONBASE: 

STRUCTURE [ (COMPRESSORBASE (COMPRESSORBASE)) 
(COMPRESSOR (COMPRESSOR) 

(REF COMPRESSORBASE TOP)) ] 


COMPRESSORBASE: 

STRUCTURE [ (BASE (BRICK 31.625 15.0 1.5)) ] 
ATTACHMENTPOINTS 

[ (TOP (REF BASE TOP)) ] 


COMPRESSOR: 

STRUCTURE [ (TANK (TANK)) 

(PUMP (PUMP) 

(REF TANK PUMPPLACE)) 

(MOTOR (MOTOR) 

(REF TANK MOTORPLACE)) 
(BELTHOUSINGFRAME (BELTHOUSINGFRAME) 
(REF TANK PLATFORMBACK)) 
(PRESSURESWITCH (PRESSURESWITCH) 

(REF TANK PRESSURESWITCHPLACE)) ] 



BELTHOUSING 

FRAME 
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FIGURE 12 GEOMETRIC MODEL OF WORKSTATION (Continued) 





















STRUCTURE [ (TANK (HORIZONTALCYLINDER 26.0 12.4) 

(MTRSP 0 0 1.6)) 

(LEFTLEG (BRICK 2.5 8.0 3.1) 

(MTRSP -8.0 0 0)) 

(RICHTLEG (BRICK 2.5 8.0 3.1) 

(MTRSP 8.0 0 0)) 

(PLATFORM (BRICK 20.5 9.125 3.15) 

(REF TANK TOP) 

(MTRSP .375 0 -2.5)) ] 

ATTACHMENTPOINTS 

[ (PUMPPLACE (REF PLATFORM TOP) (MTRSP 5.85 2.45 0)) 
(MOTORPLACE (REF PLATFORM TOP) 

(MTRSP -3.15 0 0) (MROT "Z" -90.0)) 
(PLATFORMBACK (REF PLATFORM BACK); 
(PRESSURESWITCHPLACE (REF TANK TOP) 

(MTRSP -U.5 0 0))] 

MOTOR PLACE 



<c) TANK 
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FIGURE 12 GEOMETRIC MODEL OF WORKSTATION (Continued) 














PUMP: 


MOTOR: 


STRUCTURE [ (CRANKCASE (BRICK 5.0 3.5 5.5)) 
(CYLINDER (BRICK 3.1 3.1 5.0) 
(REF BOTTOM CYLINDER)) ] 



STRUCTURE [ (MOTOR (HOR1ZDNTALCYLINDER 6.75 5.5) 
(MTRSP 0 0 ,15)) ] 



FIGURE 12 GEOMETRIC MODEL OF WORKSTATION (Continued) 






























BELTHOUSINGFRAML: 

STRUCTURE [ (BELTHOUSINGFRAME (BRICK 21.0 3.0 12.0) 
(MROT "X" 90.0) 

(MTRSP 1.1 1.5 -.7)) ] 



<f) BELT HOUSING FRAME 


PRESSURESWITCH: 

STRUCTURE [ (SUPPORT (CYLINDER 3.7 1.0)) 
(COVER (BRICK 3.6 2.7 3.0) 
(REF SUPPORT TOP)) 

(GAUGE (CYLINDER 1.0 2.2) 
(MTRSP 0 -.5 2.4) 

(MROT "X" -90.0)) ] 
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FIGURE 12 GEOMETRIC MODEL OF WORKSTATION (Continued) 



















HORIZONTALCYLINDER: 

PARAMETERS (LENGTH DIAM) 

STRUCTURE [ (CYLINDER (CYLINDER LENGTH DIAM) 

f'MTRSP (MINUS (FQUOTIENT LENGTH 2)) 
0 

(FQUOTIENT DIAM 2.0)) 

(MROT "Y" -90.0)) ] 

ATTACHMENTPOINTS 

[ (BASE NIL) 

(TOP (MTRSP 0 0 DIAM)) 

(END (REF CYLINDER TOP)) ] 



(h) HORIZONTAL CYLINDER 

SA-3B05-12f 

FIGURE 12 GEOMETRIC MODEL OF WORKSTATION (Concluded) 


Since descriptions exist only as property lists 
attached to the atoms reorpspntino model Darts, and since any cart 
mav b*» represent.ed several times in a q i v e n scene or assembly with 
different actual parameters, a function is necessary that will create 
actual Parts a* copies of the model Parts. The function CREATE 
accomplishes this In two passes throuqh the data structure. 


CRFATE must be aiven a part name from which to create 
a copy, actual parameters (if appropriate), and a position in which 
to Place the completed assembly. In the first Pass, the hierarchv of 
tne part is explored, conies of each part created, actual parameters 
























evaluated, and relative Positions calculated within each level of 
hierarchy. In the second oass, the relative positions are accumulated 
to dive an acuta! oosition for each member of the assembly. The 
primitive objects that form the terminal nodes of the hierarchy are 
passed to the SATh seament with their actual dimensions and rositiors 
for incoroor at ion into the DolVheira]. data structure. The result of 
the call on C^LATL is a copy cf the model Dart, with all actual 
parameters, sizes, ard Positions evaluated. 

vnat we havn, then, is a method of snecifvinn a 
hierarchical oraan Izat i <->n o( parts and subparts, usino some verv 
simple primitives a- the terminals. This method is more human 
oriented than previously published methods In that It allows the use 
of symbolic attachment points, and it allows the use of parameters to 
specify dimensions and other variaole information that win be 
suoplied at evaluation time. 


The data structures and functions already implemented 
could form the basis tor an Interactive parts dnsion system. Although 
we have not done so, it would be relatively easv to add facilities to 
display a continuously undated picture of an assembly as the 
proarammer or desianer specifies a new Dart, Tn fact, we intend to 
implement a minimal such facility to specify new Parts when a new 
domain for the CPC project is chosen. 

A class of narts that have not been well modeled by 
our method are nonrloid narts, such as cables and belts. To deal 
with these In the future, we orooose to implement a new class of 
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primitive called "snakes." These will consist of a cross section 
description and a set of constraints on a space curve along which the 
cross section is traced. Such an aporoach to ohlect descriptions has 
been described hy Aqin 127], Although details have yet to bp worked 
out* it seems likely that specification of the lenath of a snake and 
the positinns of its end points win be useful for most of the 
ournrses we envision. 

3. Geometric and Spatial M odels--Visin1e Outlines 

This section discusses the algorithms that use the 
symbolic models in order to identify and point to parts of a real 
compressor with the TV camera and laser pointer. Rut first* a brief 
discussion of the SAlL-ha.ed data stricture js in order. 

Information ah n ut Polyhedra is contained in short 
continuous blocks of core that we call nod^s. A node is 14 words 
lono, and contains a mix of two tyrns of data: addresses of related 
nodes, and floating point numbers recresentina dimensions, 
coordinates, or vectors. 

Correspond 1 no to each actual primitive part In the 
LISP data structure is a polyhedral reoresentation called a "body 
node." (Cylinders are approximated by eiaht-sided orisms. Pricks 
and wednes are themselves oolyhedra.) The bodies are linked together 
hierarchically to form "oblects," duplicating some of the structural 
Information of the LISP parts. Relow the level of bodies are "faces," 


"edges," and "vertices 


n a 
















The structure of a body, and its fares, edoes, and 
vertices follows the winaed-edae representation described hv Baumnart 
[?%], which -ray he summer i zed briefly as follows. 

Most of the topnloqlrai connecti'Mtv lnforTation 
about the tody is carried in edqe nodes. tach edoe node contains 
Pointers to the faces on each side of the edue, to the vertices or 
each end, and to four neiqhborlno edoes. The vertex nodes contain the 
actual x-y-z coordinates of the vertex, while the face nodes contain 
a 4-element Vector aivino the niane equation of the face in 
homoneneous coordinates. Tn addition, there j s space in each face and 
vertex node for transformed coordinates oivlnq the same information 
in the coordinate frame of the TV camera or laser pointer. fFrequenf 
mention w i11 be made this section of homoneneous coordinates, and 
of trans lat1on S , rotations, ard nrolective transformations based on 
homoneneous coordinates. It is beyond tne scoop ot this report to 
describe them. The interested reader Is advised to refer to nu aa and 
Hart r?qj. 

Associated with each ohlect Is an "encloslna sphere". 
Contained in the object node are the radius and the location of the 
center of a sphere within which the object will be entirely enclosed. 
These spheres are extremely useful in some of the aluorithms to he 
described later for minimizinq search. For an object composed of 
oniv one primitive nolvhedror, this sphere may oe calculated from t h e 
dimensions ard location of the Polyhedron. In the process of 
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combining two or more objects to form a more complex one, a simple 



geometric calculation will find a sphere (but not necessarily a 


minimal one) that encloses the union of the enclosing spheres of the 


individual oarts 


A straightforward use of the polvhedral models is in 


the presentation of a visual display of the compressor. A set of 


subroutines can convert the polyhedral renresentation of bodies to a 


set of vectors in homogeneous coordinates. The term "wire model 


appropriate here because, as shown in Figure 1) 


hidden lines and 


FIGURE 13 WIRE MODEL OF THE COMPRESSOR 




























back faces are net removed, and me dtsolay looks like a collection 
of "wires" alono the ednes of the mode). THp user has control (from 
the keyboard) of the virtual camera throtnh which t h e scene is 
viewed. Thus the dismay mav he rotated, translated, shrunk, or 
enlarqed. A SDecial split-screen v)#wor allows viowinq the lmaae ir 
binocular stereo. 


The commands to alter the display are not now in a 
form such that an apprentice co'Md pesiiv control It. But it should 
not be difficult In the future to deslqn a control lanouaae to use 
such commands as "move left," "zoom i n ," "tilt un," or "rotate." 

The alaortthms for pointinq at and identlfyinq parts 
of the compressor depend upon precise knowledqe of their Positions 
and orientations. This is because the locations of the Dolvhedra] 
models ate assumed to be correct, and reference is not made to the 
Imaqe of the compressor as such. We presume we can calculate from the 
LISP model exactlv where the imaae of each Dart would appear on a TV 
Dlcture, and makp no checks to ascertain t-hat this is actually the 
case. (We are currently enaaoed in efforts to identify the compressor 
in lmaqes obtained from the tv camera or from the laser used as a 
rame finder. This information will be used to calculate the actual 
Position of the compressor, and update our snatia] models.) 

"Cal 1hration" is the procedure bv which we calculate 
the transforms and internal parameters for the laser pointer and TV 







camera. These transforms are necessary If we are to be able to 
calculate lmaae coordinates from positions In soace and vice versa. A 
semiautomatic procedure tor calibration entails aimlm the laser at 
various places about the room where pieces of oraph paper have beep 
precisely located on the wall or table ton, and tvpinn into the 
computer tpe araPh coordinates at wnlch the beam hits the Paoer. 
Based on these locations and the deflections applied to the laser, it 
is possible to calculate (by an iterative "bill ell mb inn" alooritbm) 
a transform that oives the best correspondence between predicted and 
actual oraph coordinates. The tv camera may he similarly calibrated 
by usina the raster coordinates of the briqht snot In the lmaqe where 
the laser strikes the oraoh paoer. 

To check the corresoondence of the model, the actual 
compressor, and the TV calibration, it is sometimes useful to 
sunerimoose on the same display a diaitized Tv lmaae and the "wire 
model" of the compressor. Fiaure 14 shows such a dlsPlav. 

Dnce correspondence is established, we can find the 
intersection of a ray In soace with the compressor. This is how we 
identifv parts pointed to bv the apprentice. He points with a wand 
with a small linht bulb at its tio. The liobt bulb is identified as 
the hriohtest spot in a TV picture nf the scene. '(The laser nointer 
miaht also create a brlaht snot in the oicture, dependinq on where it 
Is aimed. But a areen filter In front of the camera eliminates this 






















FIGURE 14 WIRE MODEL SUPERIMPOSED ON TV IMAGE 


possible amblrrulty.) The stratoht line connectina the camera center 
and the liqht bulb defines the ray in space, whose intersection with 
the compressor model we are to find. 


The lioht bulb must be touchino or in close proximity 
to the part the apprentice wishes Identified. Ho maice it possible 
for him to point to the Dart from a distance would require 
determinino the position of the wand in three dimensions. Locatino 
lust the tip of the wand entails only a two-dimensional determination 
of its imaqe coordinates.) 








For intersect fnq the rav with the model, It is useful 


to wor^ in the projective coordinate system of t v ' :i TV oarer*. The 
ray, In this coordinate space, is a line of constant r and ,7 (the 
imaae coorainates measured for the nriahf snot.) 5tartfm with tne 
object node representina tne entire workshop domain, oDlects whose 
transformed enclosina srheres do not Include the coordinates of the 
ray are eliminated# and those that do are broken down Into 
subobjects, until we arrive at a set of candidate bodies that cannot 
be ruled out on the basis of their enclosina spheres. Further 
analysis of the Dolyhderal representation of each body yields either 
(1) the distance from the camera at which me rav strikes a face of 
the Dolvhedron or (7) tne fact that the rav does not strike the body 
at all. Based on these distances, the body closest to the camera is 
identified as the one Pointed to. 

fln alternate procedure is sometimes used. This 
entails subdividing the scene (as seen by the •] y camera) into a 
coarse "visibility matrix" of 40 x 40 points. Aqin and Mitzan Ho] 
describe a procedure by which a matrix representation of the edqes of 
a polyhedron may be derived, and the interior of the resultino 
outline marked or "colored in" . All of the oblects in the scene are 
sorted by their distance from the camera. Then starting with the 
farthest object, the polvhedra that make un the object are "colored" 
with a number identifyinq that oblect. Where the imaaes of two 
ohjects overlap, the number of the nearer overwrites the number of 
the farther. Fiaure is shows the result of appivim this Procedure 
to the model of the compressor, 
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FIGURE 15 VISIBILITY MATRIX 
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with this nrecomnuted matrix, determining what is 
beino pointed to by the light wan'* requires only converting the 
coordinates of the bright srot into the aroroprlafe indices of the 
matrix, and retrieving the Identifying number stored there. 

This visibility matrix arocedure provides a taster 
response to efrti Identification request, at the cost of increased 
computation during the mode 1-tuildino Dhase. In addition, the matrix 
must be recompute! each time the relative positions of the camera and 
comortssor charge. The sorting of oolerts bv distance of sphere 
centers -rom the camera is not always guaranteed to work, hut for the 
cases we have tried, the results have bpen adeauate. Thus# although 
the visibility matrix approach Is the one we generally use, we are 
aware of its limitations, and have the mope reliable rav Intersection 
method to fall pack on should cases arise wnere the faster method 
falls. 

Peoardless of w^ich procedure is used, the SAIL fork 
passes to tne I ISP fork the address of the obltct node it found. My 
referring to a table of node-part correspondences* a particular 
Drimirive part Is retrieved. There remains the Droblem of translating 
tnis primitive Into a meaningful response to the original question, 
"What is this?" A possible solution would be to-return the entire 
ancestry of the Part In its hierarchy, such as (ROOM COMPRESSOR 
COMPRESSOR-ON-BASE COMPRESSOR PUMp pUMp PISTON-CYLINDER BRICK 1 . Rut 
such an answer is unnecessarily long and redundant. For lack of a 











cet er rrode] of what Is wanted, our solution is to reply with the 
smallest oooprimitive Pa r, to w h i ch the primitive helonos, and the 
symbolic substructure name of the primitive. me above exampir 
reduces to fPUMP PTSTHN-CYM^FU). a te- special-case rules take care 
of lrstapces where such an answer would be InaoproPr1 ate or 
ridiculous. Clearlv, here is a case where models of the dialog ^nd a 

model o- the apprentice would help in formulating an appropriate 
response, 


The converse of Part 1 dent 1ticat 1 on is nojntlna. An 
apprentice may ask "where is the platform?" and expect the computer 
to point to the platform (on top of r h e tank) with the laser nointer. 
Satisfvinq this request entails two stems: findinn an outline of the 
Diattorm (takj.no into account a n v Darts that mav he j n front ot the 
Platform and hide a portion of in, and location a mint near the 
centroid of the outline at which to point. 

The first rart mav he accomplished bV using a 
visibility matrix such as the one previously shown In Figure is. 
(in this case, (.he matrix "ust bP computed in laser coordinates 
ra.her an tv coordinates, so a separate matrix is necessary.) nr 
the outline of the desired cart mav he obtained hv the method of Ref. 
31. Those Ohlects whose enclosing spheres Indicate the possibility 
of nidinn a portion of th» Platform wm p P simiiarlv outlined and 
their "colored in" nortlors subtracted fron the original. The 
procedures for recognizing occlusion and 


for subtracting the narked 
















areas are described in detail by Nitzan pi). Give', the matrix 
reorespntat 1 on of the vjsjhle rortlon of the desired oblect, its 
centroid may bp found bv the method of "chamfer 1 no, " described In 
Ref. i ?. The x and v Indices of the centroid n o t n t are converted to 
the appropriate laser deflection constants, and the laser pointer 
will then move so as to point near the center o' the nart. 

D. Deductive Retrieval Mechanisms for State Description Models 

1. Introduction 

T his section describes some modeling facilities that 
have been developed as part of the CPC. The models that we are 
concerned about here could be descriptions of any environment of 
interest at specific Instances in time. Each such description is 
said to model a "state" of the environment, and a state can be 
transformed into a new state by an event that alters the environment. 
For the CBC# these models describe the state of the workstation-- 
including the device# tools, test equipment, and so on--and the 
events are primarily maintenance and repair operations performed py 
the apprentice. 

Programming fcciiities for queryinq state description 
models ard for updating them to reflect the occurrence of an event 
are vital in many Al systems, particularly those that do question 













answerina and those that do automatic generation and execution 
monitoring ot plans. Planners, for example, use these models to 
simulate potential operator (event) sequences and to investigate 
their consequences. 

The mode 1inq mechanisms to oe described here are 
basically extensions and modifications of facilities tynically found 
in recent planning programs implemented in lanauaqes such as ClHNiVrR 
(33), PLANNER [34], and QA4 [35], Tr.e need for such additional 
knowledge representation mechanisms is evident as Al projects 
continue to move in the direction of considering mope complex task 
domains. The presentations here are meant to add an increment to 
our ability to design and build such large systems. 

?. Expressions, States, and Context;-. 

Our modelinq system is implemented as an extension of 
the ULISP programming languaqe (18). E :ach state description model can 
be thought 0 f as a set ot QI.ISP expressions, with each expression 
ha ^ ig a truth value associated with it. An expression typically Is 
a statement of a relationship among entities in the t a sk domain such 
as objects, concepts, or other relationships. For example, the 
expression (CONNFCTkd PUMP PLATFURM) is a statement that the pump i s 
connected to the Platform, and the expression (FASTENER (CONNECTION 
PUMP PLATFORM) bOLT:l) is a statement that BOLT:i is a fastener of 
the pump-platform connection. 


101 



























We use the QLISP context mechanism, which allows the 
system to build and maniDUlate a tree of state descriptions without 
havinq to create and maintain a complete codv of each state. tact 
node in this "context tree" denotes a state. To represent the new 
state that is produced by the occurrence of an event in some state 
Si, the system creates a new node in the context tree as a direct 
descendent of the Si node. All information in state Si that is rot 
explicitly chanaed in the new state is assumed to also hold in the 
new state. That is, each state Inherits model information from the 
state that is its parent in tpe context tree. 

The QL1SP context mechanism depends on the fact that 
all information associated with an expression in the data base is 
stored on the expression's property list. The presence of an 
expression in the data base implies only that there has been a 
reference to the expression (as with LISP atoms), but says nothinq 
about any of its properties, includinq its truth value, A GLISP 
expression actually nas a collection of property lists, one for each 
state in which some information has been stored about the expression. 
Hence, when an expression is asserted as being true in some state St, 
TRUE is stored as the value of the property TRUTHVALUE on the 
expression's SI property list. 
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3. Oueryina State Description Models 

a. Truth Values 

A state description model is a source of 
information about a particular situation, and its primary use is as a 
data base for answering queries about 'he situation. nur modelina 
system interacts with its users (both people and programs) as if 
QLISP expressions with truth values attached were the only 
representations being used. Hence, all queries from outside the 
modeling system concern the truth value of expressions in some qlven 
state. 

when answering a query about a particular 
expression in some given state, the system searches for a truth 
value. The search begins with the property list for the qiven state. 
If the property thuthvaluf has no value on that property list, the 
property list tor the given state's parent (in the context tree) is 
checked. The search continues in this manner until a value is found 
or until all the states in the context are considered. If no value is 
found, the search returns Unknown as its result. Since any expression 
can be stored as the value of property IHUTHVALUK, this retrieval 
mechanism allows use of an N-valued logic. For examrle, one could 
nave "fuzzy" truth values represented as integers from -100 to + 100. 
For our models, we currently are using a j-valued logic that allows 
the system to distinguish expressions that are "known true", "known 





















false"* or "have unknown trurh value" In any given state. This is the 
simplist logic that meets £ modellna system's needs since state 
description models are inherently incomplete* and it is important for 
the system to be aware of what it does not know as well as what it 
does know. 

b. Generators Instead of Backtracking 

GLISP provides facilities for associativelv 
retrieving expressions from the data base that match any given 
pattern* where a pattern is defined to be an expression that contains 
unbound variables. The QMSP statements for querying the data base 
use this pattern matching facility and are similar to the query 
statements found in PLANNED and QA4. They are desiqned to find a 
sinqle instance of a given pattern. To cause the pattern matcher to 
continue its search and obtain another such instance* the user's 
program must return to the query statement via the language's 
backtracking mechanism (i.e., by "failing"), 

Uslnq backtracking in this way to sequence 
through a class of expressions that all match a given pattern has 
severe limitations in that it ties the sequential production of each 
expression to the control structure of the user's orooram. In 
Particular* it requires that the same portion of the user's program 
be executed for each expression (namely, the statements immediately 
following the query statement). Also* since all the backtrackable 















effects of that Portion are belnq "undone" after each failure# it 
Tiakes cumbersome the savinq of results tor «ach expression generated. 
Such a back trackina mechanism is best suited to a "qenerate-and-test" 
situation wnere the us e r desires a single expression that not only 
passes the query statement's tests# but also passes additional tests 
included in the user's program. 

we have adopted the CONdiVEF solution to 
these limitations in our modelinq system bv orovidino functions that 
are generators of expressions from the data base. For example, there 
is a generator version cf the QLISP is statement called GEN: is that 
finds Instances of a given pattern having truth value THUS in a qlven 
state. Each time a generator function such as GENjis is caned, jt 
produces as many expressions as is convenient t or it. These 
expressions are put on a "Possibilities list" along with a -tag" that 
Indicates how the generator can be restarted when more expressions 
are requested, and this possibilities list lc returned by the 
generator as its value. 

if the function TPY: 1EXT is called with a 
possibilities list as an argument, it will remove the first 
expression from the list and return it as a value. if the 
Possibilities list cc.itains no expressions, then TpyjwEXT attempts to 
produce new ones by using the tag to restart the aenerator. Since 
each call to TFYjNEXT can be made from anywhere in the user's 
program, generators of this form successfully separate the production 

of a next data element from the processing that is done on each 
element, 
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Consider, 


for example, a set of queries 


concerning which components are connected to the putr^ in state Si, 
They can be initialized as follows: 

(SETO PI. (GEN: IS (CONNECTED PUMP-C) Si)). 

Then whenever one of these components is needed, evaluation of 
(TRYjNEXT PL) wm return a true instance of the pattern (CONNECTED 
PUMP *-C) and will set the value of the QLISP variable C to be the 
"found" component. 


We have implemented programming facilities to 
support the writing and use of generator functions using INTERLISP 
FUNARGS, A FUNARG is a data object that conceptually represents a 
copy of a function and a private data environment for that copy. This 
FUNARG implementation allows the definition of a generator function 
to include a set of variables (i,e., a data environment) whose values 
will be saved and restored each time the generator is restarted. 
These "own variables" allow the generator function to save pointers 
indicating where it is in its searcn for generatable items. The 
FUNARG is added to the possibilities list as the "tag" that THY:NEXT 
uses to restart the generator. Included in the implementation are 
CONNIVER-Style functions such as NOTE, AU-REVOIF, ADIFU, and 
TRYrNEXT, which make the definition and use of generators convenient 


and practical. 



























The Query Functions 


c. 


*ie can now describe our model querying 

mechanism. The following query functions are available: 

(DEDUCE:one <pattem> <context>), 

(DEDUCE; EACH <patt.ern> <ccntext>)» 

(DEDUCE;ALL <oattern> <context>), 

(REFUTE;one <pattern> <context>), 

(REFUTE;EACH dpattern> <context>), 

(REFUTE ; A L,L <patterp> <Context>). 

The DEDUCE functions find Instances of the 
pattern that are true In the given context, and the REFUTf functions 
find instances of the pattern tnat are false in the given context. 
The ;one functions find only a single instance and are not 
restartahie; the :EACH function’; are generators a n d return 
possibilities lists; and the :ALL functions return a list of all the 
flndable instances. 


Known truth values are usually not all 
explicitly stored In a model. Instead, the user provides derivation 
functions that compute them when they are needed. These functions 
may embody form theorem proving strategies or simriy be statements 
of Imp 1icationa1 rules derived from the semantics of the task domain. 





















They serve to extend each model in the sense that, from the cal lino 

program's point of view, the derived instances of a pattern are 
Indistinguishable from instances actually found in the model. 

Our query functions are similar to a lbANNER 
or UA4 GOAL statement in that they first use the pattern matcher to 
find suitable instances of the pattern in the data base and then, it 
more instances are needed, they call user supplied functions to 
attempt derivations of the desired instances. These functions are 
assumed to be generators that produce derived instances of the 
pattern. 

A typical deduction function in the CbC 
system finds and generates true Instances of patterns of the form 
(POSITIONED - X - Y) by using DEDUCE: EACH to find true Instances of the 
pattern (ATTACHED $X $Y), since components that are ATTACHED are 
assumed to be POSITIONED. A typical refutation function finds and 
generates false instances of patterns of the form (POSITIONED X *-y) 
by using DEDUCE:EACH to find true instances of the pattern (REMOVED 
$\ $Y), since components that are REMOVED are assumed to be not 
POSITIONED. 

These derivation functions are the user's 
primary means of expressing the semantic links among the relations 
occurring in the state description models. Also, they can provide an 
interface to information that is stored in representations other than 




















JLISP expressions. That Is, It fray dp much more convenient arid 
efficient to store some Information In arrays, trees, or on disk 

files? deduce and refute actions serve as the access functions to 
these alternate data oases, 

d. Storaoe and Retrieval of Action Functions 

The first element of each nonatomic 
expression in the model is assumed to be the name of a relation (or a 
3LISP variable that Is to be bound to a relation). Therefore, the 
DEDUCE and REFUTE functions can use relation names as an Index to 
determine which derivation functions should be called. Accordingly, 
we associate with each relation name two lists of derivation 
functions that can derive instances of oatterns that beqtn with the 
relation. one list contains the "deduce actions" used by the three 
forms of DEDUCE and the other contains the "refute actions" used 0 y 
the three forms of REPUTE, 

4. Savinn Derived Results 

When a model query causes derivations to be 
attempted, we want the results of those derivation- to be stored and 
retained in succeedinq states as ion? as they remain valla. In this 
way the system achieves the maximum benefit from derivations and 
minimizes unnecessary red«»rtvat ions . 


















A monel query is an attempt to find true (or false) 
instances of a qiven Pattern. Each time such an instance is 
determined/ our DEDUCE. and REFUTE query functions save the derived 
result oy assiquinq a truth value to the instance (i.e., put it as 
the value of TRUTHvAluf on the expression's property iist) so that 
the value wm not have to be rederived if it is needed aqain. -or 
example, if a deduce action for ASSEMBLED determines >;hat the pump is 
assembled by queryinq the model about each of the pump's components, 
then the expression (ASSEMBLED PUMP) will oe assidned a TRUTHVALUE of 
true. 

If a query is one of the :ALL forms, or if it is an 
:EACH form and the aeneration continues until all derivable Instances 
of the pattern are produced, or if the query pattern contains no 
unbound variables (and therefore has only one Possible Instance), 
then the system also records the fact that all instances of the 
pattern have been derived, if the same query is repeated, the system 
will know that the cCtlon functions cannot find any new Instances and 
can therefore prevent ill-fated attempts at rederivation, Eor 
example, if durlnq a query all the components that are positioned 
with respect to the pump have been found as instances of the pattern 
(POSITIONED PUMP «-C), then, when that information is requested in a 
later query, derivation functions such as the one that looks for 
components attached to the pump will not be recalled. 

These "set completeness indicators" are also 
frequently useful to Indicate the case where there are no derivable 














instances of a pattern. For example, if all derivation attempts are 
unsuccessful at determinino wnether thp pump is assembled, then the 
set of derived instances is empty and marked as complete. 

Our algorithms for matnfa!ning these derived results 
in succeeding states depend on availability of the "support" for *ach 
derivation. The "support" tor a deriveri instance is defined to be 
those expressions from, the model that are used as axioms in 
constructing the derivation. For example, if an action function 
queries the model for the locations of two ohjects and concludes that 
one of the objects is above the other, then the locations of the two 
objects form the support set for the result. Actually, since any 
model query may return a derived result, the support set tor the 
"above" result would be the union of the support sets for tne two 
location expres Ions. 

A derived result remains valid in succeeding states 
as long as its support remains valid, we therefore nave the system 
do the required maintenance on derived instances in new states by 
Including the following facility: 

Whenever an expression with a known truth value has 
its truth value changed during a model update, the 
truth value of each of the expression that it supports 
is set to UNKNOWN in the new state. 


















justification for the earlier conclusion about the pump's position is 
no lonaer valid. 


A similar mechanism is used to sav e and maintain the 
"completeness ndicators" that are attached to derived sets of 
instances. 


State Transitions 


Updating Functions 


The models of state chanaing operators that a 
system works with must contain sufficient information about the 
effects of each operator so that they can be simulated and a 
description produced of the expected resulting state. As in most 
Planning systems, we are assuming that the application of an operator 
in some state SO is modeled by producing a new state Si that is 
conceptually an updated copy of SO (i.e., Si is a direct descendent 


The truth values of tnese "supportees" may not In 
fact have changed in the new state, hut. the derivations that made the 
truth vjiues known are no longer valid. If a model auery needs to 
know one of the deleted truth values in the new state, a new 
derivation must he attempted. For example, if (ATTACHED PUMP 
PLATFORM) IS the support for (POSITIONED PUMP PLATFORM) and a detach 
pump from platform action causes a new state to he created, then the 
truth Value of (POSITIONED PUMP PLATFORM) wm be set to "unknown"; 
the pump may still be in position on the platform, but the 
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ot SO in the QLISP context tree). The effects of the operatoi are 

indicated by asserting* denying* and deleting expressions in the no* 
state Si, 

In our modeling system we provide the 

following set of model updating statements! 

(SIM:ASSERT <expression> <context>)* 

(SIMsUENY <expression> <context>)* 

(STM:DELETE <patt.ern> <cuntaxt>), 

SIM:ASSEPT (SlMiDENY) changes the truth value of the given expression 
to TRUE CFAlSE) in the state indicated by the given context. 

SIM:DELETE changes the truth value ot ail expressions that match the 

given pattern to Unknown in the state indicated by the aiven context. 
These statements also call a set of user supplied functions (like 
PLANNER antecedent theorems) that typically make additional chances 
in the new model that are direct results of the assertion* denial* or 
deletion being done. These user supplied functions play an important 
role in simplifying operator models in that they allow the user to 
express in one place side effects of particular assertions* denials, 
or deletions that always occur no matter what operator does them. In 
this way* these side effects do not nave to be repeated in each 
operator that causes them to occur. 

As in the case of the DEDUCE and REEUTh 
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functions (and for similar reasons)* we have elected to store the 
user supplied updating functions on each relation's property list. 
Hence, a relation can have a list of ASStPT:ACTIONS, dknv :AClinns, 
and DiLtTEjACTlONb. These lists indicate how a model updating 
operation should proceed for an expression having the qiven relation 
as its first element. 

The updatim functions also perform the 
maintenance operations on derived results. This means that if the 
expression had a known truth value and that value is helng changed, 
then all the expression's supportees must he deleted. Also, checks 
are made to determine if any set completeness Indicators should he 
removed. 

b. Model Updatina Usina Deduce and Refute 

Actions 

Consider now a consistency checking Procedure 
that could be applied as a standard part of model updating. The 
purpose of the procedure would be to perform additional assertions, 
denials, and deletions that are implied by the qiven expression's 
truth value change without requiring the user to write additional 
action functions. For example, if the user has written a deduce 
action that embodies the rule "X implies Y", then we do n<t want him 
to also have to write a deny action for Y that removes from the model 
any truth values that could be used to derive X. The information 















necessary to do these chanties at the appropriate time Is Included in 
the oriqinal deduce action. 

This procedure would work as follows. 'Hip 
truth value ot the expression being updated w 0 uid be set to iMKuDvt 
and an attempt would oe made to deduce the expression. if this 
attempt produces a successful derivation, then the new state contains 
support for the truth of the expression even tnouah it is beinu 
denied. The inconsistency can be eliminated by removing the support 
for the derivation. If the support sot has exactly one expression in 
it, then that expression can have its truth value reversed. (Tnls is 
the "X implies Y" case where the denial of i is Impiyind the ieniai 
of X.) The reversed truth value would be stored as a derived result 
with the original expression being upoated (Y) as its support. ivhen 
the support set contains more than one expression, we know that at 
least one of the expressions must have its truth value changed, put 
we do not know which one(s), Therefore, the best we can do is to 
delete (i.e., make truth value unknown for) ail of the support 
expressions. 

If the system knows which relations are 
changeable by events and which ones are true in all states, then it 
can decrease unnecessary deletions oy removing the unchanaeable 
expressions from the support set before considering deletions or a 
truth value reversal. After the support for the derivation has been 

removed, a new derivation is attempted and the Process is repeated 
until no new derivation can be found. 
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This updatinq procedure does not guarantee 
consistency In the new state nor does It prevent later chanaes to the 
state from Introducing new inconsistencies. However* it does 
automatically take care of many model updating details and it removes 
all exlstina inconsistencies in the state that are riiscoveraole by 
the system's deductive machinery. It the system cannot derive the 
facts from which an inconsistency follows, then the inconsistency is 
irrelevant and can safely be iqnored. 

Obviously, there are situations in which this 
procedure initiates computationally expensive derivations and causes 
many unnecessary deletions. Hence, it must be selectively applied, 
We currently have the entire process under user control hy allowing 
individual specification of which deduce and refute actions are 
employed to determine truth value deletions and reversals durlnn 
model updating. 

Note that using an "x implies Y" deauce 
action as a deny action for Y is not the same as writing a oeny 
action for Y that simply denies X. The difference is that in the 
latter case X would be denied each time Y Is denied, and the deny 
actions associated with X would then triager off other assertions, 
denials, and deletions. Such a process could clutter up the model 

with many irrelevant Implications of the dental of Y. However, in 
the former case where the deduce action is used as the deny action 

for Y, no changes are made in the model if X cannot be derived: and 






























If X can be derived, only the supporters of the derivation are 
changed. This means that only tnose truth values that are actually 
inconsistent with the denial of f are changed; no irrelevant 
implications are stored. 

6. The delations 'AND', 'OR', and 'NOT' 

AND, L)R, and NOT are "built into" our modeling system 
in the sense that deduce, refute, assert, deny, and delete actions 
have been written for each of them, whenever possible, conlunctions, 
disjunctions, and negations are decomposed into more primitive forms 
by the action functions. For example, the assert action for AND also 
asserts each of the conjuncts, and the deduce action for NOT strips 
off the NOT from the guery pattern and attempts to refute the 
remaining pattern. 

The refute actions tor ANh and OR translate the query 
into a call on DEDUCFjEACH by using the rules; 

((NOT XI) AND ... AMD (NOT Xn)) implies (NOT (XI OR ... OR X n ), and 
((NOT XI) OR ... OR (NOT Xn)) implies (NOT (XI AND ... AND Xn) . 

The deduce actions for AND and OP have a important 

role to Play in that they are the overlords ot the derivations of 
each conjunct or disjunct. They could each be expanded int^ full 

problem solving executives that would make use of co-routine 


117 










facilities to exolore alternative Derivations in parallel and 
semantic information to determine the order in which conjuncts or 
diriuncts are considerec . we have experimented with orlv 
unsopnlsticated versions of th p se actions, but the important point to 
note is that the query mechanism dives those actions control over the 
derivation so that the option is there to exnand them when needed. 

7. Summary 

we have described a set of Drodramrrinq facilities for 
building* maintainino, and queryino state description moaeis. These 
facilities are useful in systems such as planners, question 
answerers, and simulators. They allow the storage and retrieval of 
statements with true, false, and unknown truth values, and provide a 
programming environment that allows derivation rules embodying the 
semantics of a task domain to be easily added as functions to the 
system. These rules can also be used to assist in modeling the 
effects of an operator that creates a new state. Facilities are 
provided to save the results of these derivation functions, and to 
delete the results in new states where the derivations are no longer 
valid, finally, the semantics of conjunctions, disjunctions, and 
negations are provided as a part of the system. 



















Usp of Vo 1cp Input and nutout 


Tmp1ementation of VoIcp Inout Uslno thp VTP-lon 

Tb.p rHC System employs a VTP-ion voice recognition 
dpvlre which is capable of re C oqrizinq 64 utterances of duration up 
to two seconds earn, when trained for a specific voice. The proC ess 
Is as follows . 


First the messaqe data Is inout. This consists of up 
to 64 display messaaes of up to 16 characters each. Fach of the 64 
messaaes must have a unique last character, since only the last 
character is transmitted by the VTP-100. After the messaaes have been 
read from paoer tape or typed, the ViP -100 j s tralred for a 
particular voice. This process consists 0 f the VIP-ion promptina the 
person by displayina a messane, wnereupon the person speaks the wor>? 
or phrase Into the microphone, usually this is done 5 to 10 times 
for each of the messaaes. After all the messaaes have been trained 
for the person's voice, a recoanition phase Is entered In which the 
person speaks anv of the 64 messaaes, and the VTP-mo displays the 
recoqnlzed messaqe data on its 16 character display screen. If the 
utterance is not recoanized, the reject liaht will be turned on. 
Individual messaaes may be chimed or retrained, and the final 
acceptable messaqe data and trainlnq data may he output on punched 
tape for later use. 





















Fverv flue an utterance is received and recoonIzert 
durinu the recoanition phase, a character is output to the rpP-10 via 
the PDP-15, This output character is the last character of the 
16-character display cessaoe. An inverse nrocess in the PDP-io 
oroaram receives the character and retrieves the associated 
utterance, 

Table ? contains a list of the utterances currently 
used in CtiC demonstrations. fce have found this rr e 11 mlnary voice 
Innut system to he quite adequate for our current, demonstrations. 

The eventual speech *ecoqnition system employed h'» 
the rnc system win incorporate future advances in machine 
recoanition of natural lanauaoe, usim acoustics* syntax, semantics* 
discourse analysis, and so on. Meanwhile, a few interim improvements 
are Planned in the current viP-lhh system. 

wp will develop the capability to store messaae ard 
trainlnq data in the PpP-10 and transfer it (via a PDP-11) to the Vjp 
without havinq to use paper tapes. This will make it Possible to 
have a laraer input vocabulary, with certain kpy words indicatlnq 
which qroup of 64 words to use to recoanlze certain utterances. This 
will also make it possible to have trainina data for many persons 
available simultaneously* oernans with automatic selection based on 
recoanition of a few samnie utterances. 
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SHfJWMETHE 


(CONTROL Y) 


GRAPHICS 


23 

TABLE 

W 

GRAPHICS 

24 

TANK PLATFORM 

V 

GRAPHICS 

25 

INSTALL 

Y 


26 

REMOVE 

Z 


27 

CONNFCTEn 

1 


2a 

POSITIONFP 

? 


29 

LASER 

3 


30 

CAMERA 

4 


31 

DEDUCE 

5 


32 

STOP 

6 

PROC NFT & GRAPHICS 

33 

OBJFCTCENTFP 

7 

GRAPHICS 

34 

CFNTEPOF 

R 

GRAPHICS 

35 

WHAT IS 

9 

GRAPHICS 

36 

WHATS 

0 

GRAPHICS 

37 

VI SCENTER 

A 

GRAPHICS 

38 

BELT 

R 

GRAPHICS 

30 

PRESSURESWITCH GAUGE 

C 

GRAPHICS 

40 

CONSULTANT 

D 


41 

POWERCORD 

E 


42 

PRESSURESWITCH COVER 

F 

GRAPHICS 

43 

EXPFPT 

G 


44 

POINTLASFP 

I 

GRAPHICS 

45 

WHFREISTHE 

U 

GRAPHICS 

46 

PAUSE 

K 

PROC NET K, GPAPHICS 

47 

BRP AK 

I. 

PROC NET S, GRAPHICS 

48 

ROOM 

N 

GRAPHICS 
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li 

PROC NFT a GRAPHICS 

50 

START 

P 


51 

PUMp jpp 

0 

GRAPHICS 

52 

f-UN’P BOTTOM 

R 

GRAPHTCS 

53 

tank lfftlfg 

S 

GRAPHICS 

54 

TANK HIGHTT.FG 

T 

GRAPH ICS 

55 

APPRENTICE 

U 


55 

RIJMPRRACF 

V 


57 

tank platform 

1 

GRAPHICS 

58 

TANK CYLINDER 

» 

GRAPHICS 

59 

CYMNDF P 

s 

GRAPHICS 

60 

GRAPHICS 

& 

PROC NET 

61 

QUIT 

# 

GRAPHICS 

62 

CONTINUE 

z 


63 

PLEASE STOP 

t 

PROC NF T f, GRAPHICS 


Implementation of 

Voice Output Usino 

VOTRAX Phoneme 


Synthesizer 




The CPC system 

uses voice cutout 

when interactlno 

with a 

user/technician. For these 

purposes* we use 

a VOTRAX model 

VS-5 

phoneme synthesizer, which 

is a device caoahie of producina 

sounds 

corresDOndina to 63 phonemes* each with a 

choice of four 

inflection values. Phoneme and 

inflection are under the control of 



123 




/ A 

/ 

< ,% ,\ 

. y-y~ >■- v- ;■» v-y-L'-vv;,** 


















the CBC program. in addition, there are manual controls on oi tcb, 
volurrp, and speech rate. Finally, the VOTR/lX itself contains a 
built-in analysis facility that emoiovs one phoneme look-ahead in 
order to make the continuous speech sound more natural and 
lntellinihle. The result is a mechanical, hut quite understandable, 
voice to use for output in the demonstraton proqrams. 

The VOTRAX consists of a keyboard, a control unit, 
and a speaker. The keyboard is used for manual input of phonemes, 
but is not used when the VOTRAX is under computer control. The VOTRAX 
is attached to the PPP-tO through the teletype patch panel. The 
computer proqrarr sends special 8-bit codes that are converted by the 
VOTRAX control unit and output throuqh the speaker. (A future 
modification to the VOTRAX will allow it to share a teletype line 
with an ordinary computer terminal. This will be an added 
convenience, out win not chanae the operation of the proaram.) 

The current use of the VOTRAX is for output of 
orestored utterances (words or phrases). Each word is represented 
Phonetically, and this is an interestina problem in itself. The 
VOTRAX is designed to reproduce Mid-Western or standard American 

English dialect, which is the dialect used almost exclusively by the 
nationwide media. Still, it is a skill to represent words 
Phonetically (and it is a matter of opinion as to how well a olven 
word has been represented!). 
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Table 3 contains a llstino of tbe 6 3 phonemes with an 
exanpie or description of the associated sound. It should p e noticed 
that some typical Pnolish sounds (Ion vowels, for example! require 
two phonemes to reproduce them. Some of these are listed in Tahle 4. 


Table 3: VfiTPAX Phonemes and Ftoht-Rit Codes 


Phoneme 

Octal 

Tvolcal Use 

Symbol 

Code 

or Oescrlntlon 

PAO 

03 

(pause 1 

PA i 

7b 

(shorter Pause! 

A 

40 

(to aet lorn A sound use A,ayi 

A 1 

06 

(shorter than A) 

A2 

05 

(shorter than At) 


6 6 

bat 

\F.X 

6 7 

(shorter than AK) 

(H 

44 

hot 

(HI 

25 

(shorter than AH) 

kH2 

10 

(shorter than Ah l) 

iW 

75 

awful 

W! 

23 

(shorter than AW) 



















(shorter than Adi) 


(to get lono n sound use A,AY) 
hob 

-natch, chair (CH Is T,CH or DT,CH) 

-lav 

butter 

(to net Iona fcl sound! use E or 13,El) 

(shorter than E) 

ten 

(shorter than eh) 

(shorter than ehi) 

(shorter than EM2) 

her 

fire 

aet (not the c; in George) 
hay, ahead 

kit (to get long I sound use ah,fi ) 
(shorter than 1) 

(shorter than 11) 

(shorter than 12) 

(to aet long U sound use IU,U) 
let, George (j is D,J or DT,J) 


key, sick, car 




























br 1 na 


4b 
65 
64 
27 
26 
45 
53 
37 
21 
62 
71 
7 0 
50 


S2 


L 7 


5 5 


il 

l? 

! 2 
i7 


(to net long 0 sound us* MHi,on 
(snorter than 0} 

(shorter than m 
boo* 

(shorter than 00} 

Pot 

area 
see 
shv 
tea 
t hr ee 

then (the voiced Td) 

(to net long n sound use Il',U) 

(shorter than U} 

out 

(shorter than UH} 

(shorter than UMi ) 

(shorter than UH21 

seven 

won 

^ary 

(Y used as a consonant as In "vns"} 
zero 

azure (the Z) 


measure (the 5} 

















Table 4: Sounds Forced with Two Phonemes 


Sound 

Tyrica] Use 

Phoreme representation 

J 

let 

D,.T or DT,J 

CH 

match 

T,CH or DT„CH 

DW or nu 

cow 

A H l , 0 1 

01 or oy 

no j se 

01,EH3,E1 

0 

queer 

K,W 

A 

name 

A # AY 

E 

tree 

E or I 3 , F1 

I 

hlah 

AH,El 


heiaht 

A H 2 , F 1 

0 

note 

UH1,01 

u 

tH 

IU, 11 



The 

octal Phoneme code 

Is 

six 

binary hits 

of 

information. These 

are preceded by two 

bits 

representlna 

the 

Inflect Ion 

level. In 

practice, the Inflection 

code 

Is represented 

as 

eloht bits 

(with the 

last six beinq zero), 

and 

the 

code sent to 

the 

VOTRAX Is 

the sum of 

Inflection and Phoneme 

codes, 



Inflection 

Levels are 






1 

30n 

(lowest) 





2 

200 

(normal/default) 





3 

1 00 






4 

000 

(highest) 
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The null code, which is used for margin end of transmission to the 
vnTPAX, is the octal code 377 (binary ltlltlin. 

To date, tbp CBC oroipct has used approximately rod 
phonetic words. Many were copied from a list suoDled by the vpTRAX 
manufacturer, and nearly the same number were incut hv various users 
of the eciuipment. A MSP Drooram has been written to handle the 
storicq, retrievtnn, and transmission of Phoneme representations. A 
few tvcicai representations are reproduced below. 

(PLEASE (2 P)(l f, )(l f)(i Ej 1(2 Z) ) 

(ASSEMBLE (2 UHl)(? S)(2 EHJ)(2 M) (2 M) (2 R)(2 ”H3)(? L)i 
(AIR (2 AE1(1 ER)) 

(COMPRESSOR (2 K) (2 HM1)(2 M)(2 P)(2 R)(2 EH)(1 S)(1 R)(1 R)) 

Addino to the dictionary is simole, via the LISP 
function ADhw. An example is explained below. 

AT hW(TESTING) 

This line is typed by the user. 

TESTING! 

This line is typed by the proaram which then waits 
for the user to respond with the list of inflection and 
Phoneme pairs. 

2T 2FH 3S 2T 1NG 1UH3 

The user inputs a try at the Phoneme representation. 

The VOTPAX pronounces the input strino, then the pronram 
types OK? and waits for user response. 













£ 


The user has a choice of four possible responses. 

If the user types Y, the word win be accepted In 
Its present phonetic fore. 

If the user tyoes R, the word win be spoicen aoaln. 

If the user types N, the word is not acceptable and a 
new list of phonemes must h® trout. 

If the user types F!, the word is not acceptable hut the 
user is allowed to edit the previous list of Dhoneees, 


This process recycles until the user types "Y", at which time the 
word Is added to the dictionary list and stored In the special data 
space described below. 


The storaae of phoneme representations of words maices 
use of LISP's ability to swap data Into an Inferior fork. In this 
case# we use the for* Into which MSP swans comniled code. The snace 
for the Phoreme data is a lane swaoppd array. Phoneme and Inflection 
pairs are stored four to a word# and a pointer Is kept of the address 
of the last phoneme stored. When a new word Is to be stored# the 
pointer Is incremented by 1, the value of the pointer Is stored in 
the system hash array In the hash address for the word# and the fi-blt 
codes are stored with an additional leadlno "0" (mariner 9-blts each! 


sequential 1y Into the swapoed data array. The last code stored 
contains a "1" In the left most bit# slonallnq that the end of the 


word representation has been reached 














a '"i" ir> to oe spoken, the "handle" to the 
phoneme representation is obtained by GFTH ASM (word) and the list of 

Dhoneme codes is retrieved fr om the swapped data space and put into a 
buffer (a LISP array) to he sent to the vdtrax. The capacity of the 
buffer is 34 words or 136 Phonemes. A pointer is kept to the current 
end of data in the buffer* and many words and nhrases may be stored 
into the buffer heforp they are actually transmitted. Transmission 
to the VOTRAX (across the teletype lines by the tf.nfx operation 
System) takes Place either when the buffer is filled* or when the 
oronram determines that no more words are to be utterred. The end of 
transmission is signalled to the VpTRAX by placinn a code "377" at 
the end of the data in the buffer. The VDTRAX itself has a buffer. 
Voice output is initiated hv the VOTRAX when its buffer is half full. 
Thus, if is possible r 0 se n d a second transmission to the VOTRAX 
before the buffer is completely emoty* makina continuous speech 
Possible. 

Provision Is made for Savina and restorinn 
dictionaries so that phoneme representations must only he input once. 
The LISP functions MAKFriCT and LOADPICT accomplish this task. They 
dePend on a list of words b«»ina sav e d as the vaiu e of the LISP atom 
called "PTCT". (in demonstration proorams* w p frequently set PICT to 
NIL to save the snace required to link an the dictionary w 0 rds into 
a slnqle list. This means* however* that we cannot save an nodated 
version of the dlcMorarv; but this is not usually a hindrance.) In 
the case of MAKFPICT, a disk file is opened* and each word on PICT is 



























written on the file with its correspondino Phoneme list (not the 
binary form--rether the form that exDlicItlv contains infjpctlon 
numbers and Phonemes). LPAPDTCT operates in revprse* ooenlnn a 
specified disk file for input, readino sequential entries# convertino 
Phonemes and inflections to binary codes, and storino them i> the 
swapped data area. 

The use of computer voice output has been very 
satisfactory in all the demonstrations we have conducted thus far. we 
intend to continue to use the VOTRAX f 0 r the forseeabie future. 
Refinements will be made in the renresentations of exlstina words and 
new words wm, 0 f course, be added. A proqram will be written that 
will attempt to make its own ohonetic representation of a new word. 
Another new proqram will provide tor dynamic inflection of a whole 
utterance, which win he a oreat improvement over thp current method 
of prestorina the Inflections for each individual word. 

F. Svstem Inteoratlon 

The ultimate form of the Computer Based Consultant Is 
expected to be a sinale comouter nroqram which may be, in fact, a 
combination of m a nv different, put interlinked, computer programs. 
The system bulldino task is a bio and important one and there are few 
precedents or theories for bow to do it. The main philosoDhy that has 
qulded system buildina of the CRC up to now is that as different 
oroqrams and "abilities" become available (even if thev are only in 

















preliminary nr simple form) they are included in the system as soon 
as possible, Ttvls anrroach has several advantages. 

The main advanfaae Is that we can net a "flavor" of what the 
eventual system win be like. For example, an earlv system was lust 
the procedural net rroaran, expanded with air compressor semantics 
-^nd knowiedoe about tools. [t soon oecame apparent that voice inout 
and output would he "nice", so that was added bv takino advantaoe of 
available devices tn a t use preselected words and Phrases f 0 r inout 
and outout. 

Thinqs that are seen as deficiencies by casual observers of 
the demonstrations are taken as important next steps for the system 
builders. For example, it soon became noparent that manv questions an 
apprentice would loqically ask, such as why?, where?, what if , 
and so on, could not be handled. The mechanisms for a few of these 
were coded in rudimentary form so that their effects could be tested. 

Another important consequence of continual svstem buiidinq is 
learnina where different. Darts of the system overlap. For example, 
several different "models" a r e already m use CI.e., a connectivity 
model for deductive question answering, and a oolvhedral model for 
graphical display and pointing). We must ask whether the models can 
be combined. fire tnev compatible? Can all models he kPPt up-to-date 
simultaneously without too much trouble? wh»re ran one program net 
information from another program's model? 









































In summary# the current (April 197*5) system has the followinn 
capabilities: 

(1) It knows about top level components of a specific air 
compressor, 

(2) It knows about some basic hand tools (screwdrivers, 
wrenches, alien-wrenches) sr.ri fastenlnas (screws, 
machine-screws, nuts and bolts, set-screws, washers), 

(3) It knows about assemhly and disassembly sequences 

(pos Itlonlnn, connection, fastening, removlnq, extractina) 

(4) It kno*-' about parts adjacent to other parts, both In toe 
connectivity (assembly/dlsassembly) model and In the 
polyhedral (araphics) model, 

(5) It can use a laser beam to point at the location of a part 

(6) It can determine which Part Is indicated by a 
1 laht/pointer or laser beam, 

(7) It can use voice inout and output in a simple form, 

(8) It can create Plans for accomplishing tasks and answer 
auestlons about the plans, 

(9) It can execute the Plan (l.e., tell a user how to Perform 
a task at levels of detail based on the user's varying 

responses), 

(10) It can answer auestlons about the chanced state of the 
equipment at any time, 

(11) It can perform simple execution monitorlna of the 
proaress of the apprentice In executina a plan. 




















U?) It can perform calibration of thp laser-ranoefJnder ana 


the TV camera In a sc»j =>nto^at 1 c mode. 


This is an impressive list of capabilities for a rroqram that 
Is still In Its early staaes. Fach of the capabilities needs more 
development, and certain important capabilities are Included only in 
rudimentary form (such as execution monitoring, natural lanauaqe, and 
vision). Still other capabilities such as troubleshootina are not 
included at all. Yet, bv making an integrated system out of 
available parts we are able to envision the final system much more 
readily and recoonize whe r e our efforts should he directed. 

The "top level executive" or driver program for this system 
is expected to be the execution monitoring Portion nf the procedural 
net (NOAH), because this is the program that carries on the basic 
interaction with the user. This may be closely associated with the 
eventual speech understanding program, with the semantic net model 
becoming one of the primary sources of information about the task 
(and world) domain. 


The system is currently Implemented oy taking advantage of 
the fork structure under the TKNEX operating system. The main 
programming languages are MSP and omsp, wjth use . 0 f SUL, fortran, 
and machine language where appropriate. The fork structure is shown 


in Figure 1 f >. 
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FIGURE 16 STRUCTURE OF THE CBC DEMONSTRATION PROGRAM 



























III. COMPONENTS OF SHHSKOlIFNT SYSTFMS 


A. Introduction 


In parallel with ronstruct1nn the present system, we nave put 
In a oreat deal of desian work on components of future ('91h thrmqh 
1^78) CMC systems. In Section IIT.H we oresent a detailed description 
of our wor* to provide a powerful natural lannjaae understandlna 
system, our Intention Is that beginning in 1Q7b we will he able to 
process some text innut to the svstem. (Th« text input system is 
belna desianed to he part of a svstem that will ultimately handle 
speech.) Durlna 1977 we olan to make extensive additions to the power 
of the text system, and finally In 1978 we expect to extend it to 
handle utterances spoken by the apprentice. Gary Hendrix and Barbara 
Deutsch have been responsible for our natural lanouaoe research 
working in conjunction with the SRi speech understandlna project. 


In Section Itl.C we describe two pieces of additional problem 
solvina work. First, we nreSent some research Performed bv Marty 
Rattner on how to model and compute the "freedom of movement" 
properties of equipment. Next w e discuss the preliminary results of 
Richard nuda and Mis Mlsson toward buildina a system for diagnosis 
of euuipment malfunction. 
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Our work In vision Is described in Section TIT.n. It will 


ultimately Drovtde the capability of answering questions about the 
equipment and the workstation by direct observation. The vision work 
represents the combined efforts of several people, notably f^artv 
Tenenbaum, Harry Harrow, and Thomas Garvey Cdoinq Ph.D, dissertation 




work). 

B. Natural Languaqe 

1, Introduction 

The goal of our work in languaqe is to provide a two 
way communications link between the consultant and the apprentice 
that is both natural and convenient. We are persuaded that the most 
appropriate communications medium is that of spoken English. The 
reason for this is twofold. First, speech trees the apprentice from 
the burdens of Interacting mechanically with the consultant while 
performing his tasks. Second, the. use of Fnglish makes it 
unnecessary for the apprentice to cope with a formal language, thus 
allowing him to concentrate all his efforts on the performance of the 
work task. It is anticipated that by 1978 the lanquege component 
will have the ability to understand the user's spoken questions and 
remarks regarding the state of his task and the use of tools. 
Further, the system will have the ability to produce spoken responses 
to verbal inputs. 
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in developing a technology tor the understanding of 
continuous speech, our research efforts are being carefully 
coordinated with those of the SRI speech project [36], As a result 
of cooperation during the past year, a common oaslc grammar and a 
common semantic Interpretation system have been developed, further, 
the focused parser desianed for the speech project has been adapted 
for (Interim) text processing by the CBC. while various acoustic 
processing techniques are being developed, this parser win allow us 
to conduct experiments durlnq the next two years uslnq text and win 
facilitate the eventual conversion to acoustic Input in 1978. 

In keeping with the qoal of providing natural 
communications between consultant and apprentice, we have, during the 
last year and a half, collected and analyzed numerous protocols 
between human consultants and apprentices, within the past year, this 
analysis has aided us In the selection of a working vocabulary and in 
the development of a tentative performance grammar (which extends the 
grammar developed jointly with the speech project). Having used the 
protocol experiments to clarify the language requirements of a 
computer based consultant, we have been developing a system 
architecture and a semantic representation scheme capable of dealina 
with the linguistic problems likely to arise in consulting 
situations. Details concerning each of these achievements follow. 
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Protocol Collection 


We have extended the collection oi protocols 
described in our previous annual report C 3 7 3 to inciuae different 
tasks and apprentices at different skill levels. One new task has 
entailed working on a subcomponent of tne compressor# namely the 
pump. This task introduced new vocabulary and, more importantly, the 
need to describe and understand questions about more complex tools, 
parts, and operations. A sample from one of tne pump assembly dlaioqs 
is shown in Figure 17. 

We have an experimental setup now that allows us to 
record new protocols easily, we plan to run additional experiments on 
new pieces of equipment. Tasxs for these experiments will entail 
troubleshooting as well as assembly and disassembly. 

3, The Concordance Programs 

As an aid to our analysis of tne lanquage used in 
expert/apprentice situations, we have developed a package of 
concordance programs, A sample output from these programs, 
illustrating several of the features of tne concordance produced, is 
shown in Flqure 18, This sample shows a collection of results from 
concordances done separately for several dialogs. (The dates were 
added to make the separation clearer.) For each of the that which 
occur, the concordance shows a frequency count and a list of the 
different contexts in which that word appears. 
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APPK'E.’U'ICrj'JK, isHAr NEXT? 

00*59*44 

00*59:51 

CONSUL 1 A*TjuLmCE T-t, CYLlUFR HEAD GASKET 00 THE TOP OF THE CYLINDER 
Cl* 'CM? 

Cl 100*2* 

U * WHAT Orjr'S THE CYLINDER HEAD GA5XF1 LOO* LIKE* 

01*00*36 

01 *nc*3H 

C*Tnfc CyM'O’tR HEAD GASetT IS A S-JUftHe PAPER SHEET KITH A LARGE ROUND 
HOLE IN IHt CEHTEP, ACT 4 RDM HOL^S ON THF OUTSIDE. 

01 * a 1*31 

01 * 01*44 

U*PK, I »nu.c IT, I*VE FLACer IT, *HAT NEXT? 

01*01*52 

C l *01*54 

C * 1 '-STALL T«t CYLINDER HEAD, 

01*02*07 

01*"2»1R 
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01 *o2:?2 

01 *,?:?*> 

C*TrtP IC’TAH^ ».-PEAlHfcH SHOULD HE £H YOUR RIGHT AS YOU FACE THE 
Ti*fc CRAM'fihAFT, 

"1 J03*?l 

01*03*45 

U*0, IS Ihjs e> rGHT? 

01*03*49 

M *04*02 

C*NCi# YCU. *1AVE IT PACKHAPDS, 

01*04*09 

01 * 04 * 2 ? 

I'M-fM Tf'iC f.f T USE TO ATTACH IT? 

Cl*04*28 
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FIGURE 17 SAMPLE PUMP ASSEMBLY DIALOG 
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Knowing the context in which a word appears aids In 
determining both the different semantic and the different syntactic 
uses of the word, toe nave used this information In constructing our 
wording concepts and our working grammar. By pulling an of the uses 
of a word together , the concordance enables us to determine which 
word senses are used most often. It would be difficult to get this 

same Information from the raw dialog because the word uses are so 
spread out. As an example, consider the uses of the word "all" shown 

at the top of Figure 18. These are representative of the uses w e 
found. in only one case Is "till" used in the quant 1ficatlona1 
sense (In the FEH12 dialog, ", . . I've got all four bolts In 
place 1 ). This result (and similar ones tor the other quantifiers) was 
unexpected. 



The number 

to 

the 

left 
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each entry in 

the 
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is the line 

number 
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where 

this 

entry 

appears. 

This allows 

us 

to 

examine the 

total 
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of the 

word 


use. We have used this to distinguish expert and apprentice word 
uses. For example (see Figure 18) the use of "bolt" as a present 
tense v e rb comes exclusively from the expert. The apprentice uses 
the verb form of bolt only in the past tense. 

It is possible to merge the concordance results for 
several dialogs. We have obtained a merged concordance for all of the 
dialogs, which enables us to distinguish indivic ai differences from 
jeneral trends. Those words and syntactic structures that appear 























across several dialogs are Incorporated in our working vocabulary and 
working grammar. Later# individual modes of expression will be 
handled in the user model. 


4. Dialog Analysis 

As mentioned previously, a major motivation in 
collecting the dialog protocols has been to determine the language 

requirements of the CBC. Analysis of the collected dialogs has 
proceded on several levels. On the most basic level, we have used 
the protocols in establishing our workina vocabulary and grammar. 
These are discussed more fully in separate sections. In both these 
undertakings, analysis ot the dialogs aided us not only by 
illustrating what language usages there were but also by making us 
aware of what problems did not arise. For example, as noted before, 
we were surprised by the infrequent use of quantification by the 
apprentices. Quantifier words, like "all", did appear in apprentice 
statements, but mainly in Phrases like "is that all" meaning "have I 
finished?". 


There are many well known discourse level problems in 
natural language understanding; for example the problems of resolving 
reference and completing partial utterances. In the CBC domain there 
are also the closely related problems of describing and understanding 
descriptions of objects and actions, we have been using the dialogs 
as guides for our research on these more global discourse level 


problems. 




























We have been interested not only in the forms of 
these CBC discourse level problems, but also in discoverino what 
inherent information there was in the dialoq context that would aid 
in "solving" the problems. 

The dialog context is actually a composite of three 
different component contexts: a verbal context, a task context, and 

a context of general world knowledge. The verba) context includes the 

history of preceding utterances: their syntactic form, the objects 

and actions discussed in them, and the particular words used. The 

task context is tne focus supplied by the tas< Deing worked on. It 

includes such things as: where does che current subtask fit in the 

overall plan# what re its subtasks# what actions are likely to 

follow# and what objects a r e important. The context of general world 

knowledge is the information that reflects a background 

understanding of the properties and interrelations of objects and 

actions; for example# the fact that tool boxes typically contain 

tools and that attaching involves some kind of fastening. 

An important aspect of the reference problem is 
determining what sources of knowledge should oe accessed to resolve a 
reference, Decisions must be made concerning how much effort may be 
spent testing one antecedent candidate and how much effort may be 
spent investigating the different context perspectives from which 

that candidate may be viewed. 
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In the context of 


"Tighten the setscrews with an alien wrench" 


consider the question 


"Where are the setscrews?" 


The phrase "the setscrews" must be resolved as the previously 
mentioned ones. T»is resolution comes from the verbal context (or 
dialog history). However# in the context 

"Attach the pump pulley next" 

the question "Where are the setscrews?" can only be understood if 
the consultant is aware that installing and tightening some screws is 
part of the operation of attaching the pump pulley. The resolution 
comes from knowledge of the task. Any screws mentioned in the 
previous dialog would probably be irrelevant. Finally# if w e consider 
the context 


"I have tne parts box#" 

the resolution can be found only by knowing that screws are typically 
stored in a parts box. 
























The reference resolver must consider as candidates 
for antecedent not only objects and actions that are explicitly 
represented in the dialoq history (which would work only for the 
first of our examples), but also the Interconnections of those 
objects and actions in the task domain and in the "oeneral real 
world". This means deciding which Kinds of connections to consider 
first and how long to investigate them before looking at others. It 

also entails deciding how much effort to put into looking at all the 
connections of one oblect or action before considering others. The 
implementation of the reference resolver is being desiqned so that w P 
can easily experiment with aifferent strategies for looking at the 
various contexts. The separation between the three context 
components is made explicit. Tne task context is supplied by a 
connection to the procedural net. The difference between the local 
verbal context and real-w 0 rid knowledge connections is reflected in 
the way the semantic representation of the discourse history is kept. 
Essentially, the local context is kept separate from qlobal 
knowledge, but contains a few links to that knowledge. How this is 
done will be discussed more fully after the semantic representation 
is presented. 

The problem of object description is closely related 
to the reference problem. This is obvious since the description 
problem is basically the inverse of the reference resolution problem; 
an object is unambiguously described if the description given can oe 
used to locate a unique object. Any object has a multitude of 















attributes. Some are simple (e.g., color, shape) and others involve 
connections to other objects (e.g., on-top-of, inside). However, at 
any one tine only a few of these properties are needed to uniquely 
specify tne object. This is because context limits tne other objects 
from which any one object needs to be distinguished. As an example, 
consider the situation when the apprentice is using a l/2-inch 
box-end wrench and a 1/2-inch socket wrench to tiqhten a nut/bolt 
fastening. The two wrenches may be distinguished by ty^e: "the 
wrench" is ambiguous, but both "the box-end wrench" ard "the socket 
wrench" are unambiguous. However, if the apprentice is using two 
1/2-inch box-end wrenches for his task then they need to be 
distinguished by other criteria, such as which is on the nut. 

We have been designing the discourse history 
component of the system in a way that will allow us to know which 
attributes of an object are most important in a given context. Again 
this is closely tied to the semantic representation and will be 
discussed in Section 9 below, 

5. Selection of working Vocabulary and Working Concepts 

The concordance proarams have aided us In finding 
which words are likely to occur in the workstation. Surprisingly, 
only 620 different words were used in all of the dialogs, liur initial 
working vocabulary of approximately 650 words includes some 
additional words we know will appear as new tasks are added Ce.g., 


some part names) 









s a tirst step toward hollaing a semantic 
representation of the Knowledge base necessary to understand CBf 
related dialogs, we have divided the vocabulary into conceptual 
categories. (Our semantic representation is discussed in Section R 
below. For present purposes, it suffices to Know that oblects and 

actions are represented by nodes in a semantic network. Various 
hierarchical relationships between categories of oblects and actions 

are also represented. In addition, it is Important to know that the 
result of parsing an utterance is the production of a semantic 
network.) At present there are 21 conceptual categories. The 
categories, along with a sample of the words in each category, are 
shown in Figure 19. 

There ar e several characteristics of tnls category 
list that will be important in building our network representation. 
First, the categories are hierarchical. For example, in the category 
of location operators there are general Position change w 0 rgs (e.g., 
adjust) and also position change words whicn specify that a tool be 
used (e.g., pound). Note that some of the nierarchies that are 
important in the ChC domain are not shown in this category list, 
because subclasses are not shown to ail levels. (To do tnls would 
require showing the compiete^semantlc net). Cor example, under tools 
we do not show the hierarchy tool-* wrench-* box-end wrench. 

Note that some of the categories are guite closely 
related. The best example of this is the set of categories: 
location, location operators, locate. The location category contains 
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those words that express the different pieces end positions In which 
en object »ey be found. The location operators are those words that 
have to do with effecting change (or no change) on those positions. 
The words In the locate category have to do with whether or not the 
location of an object is known. The representations corresponding to 
the words In these categories win be closely linked. 


Zn addition to the categories that have to do with 
objects (e.g.» tools) and actions (e.g.» fasten) in the domain* tnere 
are categories that have to do with expert/user Interaction (those in 


a.the.tNrse 

re,dc 

a 1 s n • " . n c t»e r 

car* rust »c*ir<<«ps#5'''"ilr: 

r.et. 

rv» tsr * to#tooctrer# * J et 
*1 l.rotr .eltrer.se* 

eecause.lt #tt 

w r. at. »• her*, wrier, »* r v 

lt#tret»they#r.Mna 

use 


assume, cert air. .tri«< 
ck.realire.understara 
cell.lcentlfy.BOlrr 
declee. •'ere t w ine.test 


152 


R.pr«a«K*a if*"* 
bail avail* bl* copy- 





tne communication category). The presence of tnis category indicates 
the importance of the subtask of communicating information between 
the the expert and the apprentice. 


There are also categories that in essence concern the 
syntactic structure of individual utterances. The linguistic 
operators category contains that which win not themselves have 
specific representations in the semantic net. The words in this 
category help quid* the construction of the semantic representation 
being Duilt up for an utterance, not by adding new semantic 
constituents, but rather by indicating the relationships between 
existing constituents, for example, the prepositions help supply the 
mapping between surface cases and deep conceptual cases. 


b. 


The Working Grammar 


Through cooperative efforts with the SRI speech 
project, a basic Performance grammar applicable to a wide variety of 
tasks has been developed. For use in tne CBC domain, certain 
extensions to this basic grammar have been made to account for 

syntactic constructions that occur with higher frequency in 
consultation dialogs than in ordinary conversations. The grammar is 

based on a thoughtful analysis of our protocol experiments, and thus 
reflects a major portion of the language actually spoken by 
apprentices in the performance of tasks rather than some abstract 
theory of how the apprentice "should" soeak. Thus, the grammar can 
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handle isolated phrases and clauses as well as complete sentences. 
Although it reflects the language of this task domain# the grammar 
does include the kinds of linguistic constructions handled in 
contemporary computational linguistic research and was written with 
extensibility in mind. 

Since the grammar is ultimately to be used in tru* 
processing of speech# it has been designed so that advantage may be 
taken of knowledge available from whatever sources are capable of 
reporting information at any given point during an analysis. The 
grammar is written so that it is not restricted to a particular kind 
of parsing strategy. That is# it can be used both bottom up in 
building more complex grammatical structures from words that have 
been Identified acoustically, and top down in working from predicted 
grammatical structures to the words they contain, (The same grammar 
should also be serviceable for response generation, although not much 
itudy has yet been made in this area.) The grammar incorporates in 
its rules information on semantic# pragmatic# and prosodic features# 
as well as on grammatical ones. Experiments conducted by the speech 
group life] show that when the basic grammar is compiled with a 
lexicon into the internal representation used by the parser, the 
result is a language processing system whose sources of knowledge are 
highly integrated and well coordinated. 


Both the basic grammar and the CBC-speclfic 
extensions are expressed as collections of rules (following Paxton 















[38J). As exemplified by (simplified and edltea) rule SI of Figure 
20, a grammar rule nas three principal components: a context-free 
production, an attributes statement, and a factors statement. while 
each rule contains a context-free component# it must be emphasized 
that the rules themselves are not context free, since the application 
of a proauction is constrained by restrictions encoded througn the 
attributes and factors statements. 


RULE,DEE Sit 

CF-PRODUCTIO 

S s Nip VPj 

AT TRIBUTES 

FOCUS, ROOD FRO" ,np, 

VOICE « "ACT, 

SEMANTICS s SERRSl(SEMANTICS(*P), 

SfiVAMTlCSCVP)), 
AFFnEG s IF ROOt) EO DEC THEN "AFF- 
FACTORS 

IF VOICE: (VP) EG PASS THEN OUT, 

IF GIN TERSECT 1 ON'(NfiR (rjp) # r,‘RR(VP)) 
THEM OK ELSE OUT, 

IF TPA*S(VP) FO 0 lrlEiN BAD, 

IF SEMANTICS EG NIL THEDvI TUTj 

FIGURE 20 A SIMPLIFIED GRAMMAR RULE 


The context-tree production ot a grammar rule 
specifies a string ot utterance constituents (either terminals or 
tokens) that, under favorable circumstances, may oe combined to form 

a larger or more complex constituent. For example, the context-free 
production of rule si is 



S 


NP VP 









which indicates that an S constituent of an utterance (a complete 
sentence or clause) may be formed by joining an NP (noun phrase) with 
a VP (verb phrase); nowever* various attributes of the np and VP must 
be in aqreement. 

To build up internal descriptions of parsed phrases, 
each rule augments its context-free production with an attributes 
statement, Lach constituent type (e.g,, the constituent types hP, VP, 
S, U used by the grammar is associated with a set of attributes whose 
values help to characterize particular instances of the constituent. 
For example, the attributes VCi ICF and MOUD help to characterize 
particular instances of vps and 3s. The attributes statement of a 
grammar rule assiqns values to attributes of the phrase being built 
up. These values are derived primarily from the values of attributes 
in the phrase's constituents. Thus, for example* the FOCUS and MOOD 
of an S constructed by rule SI are taken from the FOCUS and MOOD of 
the NP that is the first constltutent of the S, Attribute-values 
for the most primitive components of the grammar (such as individual 
nouns) are specified by the lexicon. 

To constrain tho application of rules to appropriate 
situations and to aid the parser in deciding which rules seem most 
likely to yield good results* each r >le contains a factors statement. 
Since a given context-free production is applicable only if Phrase 
subparts are compatible, the factors statement may apply tests that 
examine the values of constituent attributes to determine their 

















coTipatloi11 ty. For those phrases that pass the compatibility test, 
the factors statement derives a number, called the rule score, which 
indicates how well the constituents tit together. This numoer is 
used by the parser in determining how much effort to devote to using 
the phrase produced by the rule in suosequent computations. 

lo use a rule such as Si in the Dottom up mode, the 
parser first finds a string of the form "NP VP" to match the right 
side of the context-free production. with such a string found, tne 
Parser beqlns to check the factors. First, a check is made to see if 
the VOICE of the VP is PASS (passive). Passive voice is not allowed 
in this rule (there is another rule, S9 In Figure 21, that should be 
used for passives), and hence it VOICE(VP) is In fact PASS, then this 
application of rule si is thrown OUT and a record of the aoorted 
application is made to prevent future attempts. If the voice test is 
passed, then a test is made to see if the number attributes of the NP 
and VP are compatible; e.g., "tney mesh” is OK out "it mesh" is out. 
The passing of this compatibility test is regarded as a moderately 
° 00d sign (indicated by "OK") and the rule score becomes slightly 
enhanced. The next Portion of the factors statement checks to see 
if the TRANS of the vp (the number of yet unassigned standa-d case 
arguments) is 0, If TRANS(VP) is in fact 0, this is a BAU sign and 
detracts from the score given the resulting S constituent. 

When other tests are passed, the SEMANTICS attribute 
of S is computed from the SEMANTICS of NP and VP by a call to the 


157 























PI’LL 


COfcTLXT-FRSK-STATK -1ENT 


lio 

U s "UKAX 

U1 

II s S 

U2 

0 = NP 

U3 

u s f4C.«* 

si 

S s OP VP 

s? 

S S up AUXO VP 

52 

S * HFjNPl AUXH VPJNP2 

54 

S s VP 

S5 

5 = AUXO VP 

S6 

& s MpjNPl AUXO IMPJ-MP2 VP 

S7 

S * AllXL' HP VP 

SB 

5 s AUXB NPsMPl 'IP: , JP2 

59 

5 s NP AllXh VP 

S10 

S s A• • XP* fcP VP 

SI 1 

5 s NP AoXb AD.j 

512 

S r AUXri HP ADJ 

SI 3 

S s "HOw AOJ Al‘XB NP 

Sl% 

S = "THERE AOXc i<P 

Sib 

j> s "icLPf. AUXO NP PKEPP 

516 

S s Np AUXB -THEHF 

SI 7 

s S UP AUXB "THERE P3FPP 

SI B 

S s AUXP "THEFE NP 

519 

j, s AllXB "THERE VP PPKPP 

01 

AUX) s DO 

02 

A0X0 s OC "NOT 

L’3 

AUXO * DO -PEG 

B1 

mIIXB * BE 

B2 

AUXO a PL "NOT 

b3 

AUXB * PE -MEG 

Nfl 

NP * Hi on 

OP 2 

Up a UUMBF.PP 

MP3 

MP a NbPOLRP "OF IVP 

NP4 

NP a UUMBEHP l*oM 

NP6 

NP * DEI 

NP7 

NP s DEI "OF NP 

NPB 

NP s OFT NOR 

NP9 

ilp a DET NUMBER "OF NP 

NP10 

NP a DET NUMBER MOM 

NPU 

NP a OET NUMBER 

NF12 

NP ■ QUANT NUMBER "OF NP 

NP13 

NP a QUANT "OF NP 

NPU 

NP ■ QUANT NUMBER 

NP19 

NP • QUANT 

NPU 

NP a QUANT NUMBER NOM 

NP17 

NP a QUANT NOM 


Speech 

speech 

speech 

speech 

speech 

speech 


FIGURE 21 CONTEXT-FREE PRODUCTIONS OF BASE GRAMMAR 


• V V V w “ 

• A w> -* 





r 




WO Ml 

NOf s WQMHt.Au 


N0V.2 

NOh s AIM l<UM 


WK 1 

t:(.n'!rlF,AL s ftU.MHEAD 

PPEPP 

WH2 

rlOMH&Al' s vCHJu 


.11: J 

.'itjf.HEAO s WUMHtAu 

S 

•IH4 

MJMHEAD s lOMHEAU 

VP 

•il 

imJU'J s I 


M2 

: HIIM s „ -PLUhAl. 


;i3 

f.CHifJ s tLASSIFlit 

Li U •* 

W4 

CLASSIFIER = VP 


!■*> 

C LA SSI F'l EH s CLASS 40 * 


CLASSWO? s ADJ CL ASS 40 

*7 

C l- S S f' u j« s CLASS It* 1F A 

i>8 

C L4SSi-.il K s i> 



cue 

etc 

r^c 

ere 

Cnc 

etc 


VF'l VP S VEpr. 

VP? Vp s VP P 

VPJ VP s VP PKEPP 

VI VEP9 s V 

V? VfPo s V -SG 

vj vfhh s v -ppi. d istt'^r t icif 1? 

PPEPP1 PKEPP s PkfcP \P 


t't. T 1 I t I S M -Gh"i 


home l 

MK3KRP 

S "HiJ* Ap 

MirfPZ 

•'ii> h !dfc:pp 

S 'vP 

MHP3 


s I H A :>iH "IHA'i -itMh 

uU : ’P4 

•■il* -lr£r<»- 

s Ml Mb EH 

Ml Ml 


s Sl-iAT Lk’.I/ 

NOr ? 

mjmhfh s 

SMALL -JlM "HUWPHbO 


"A.*0 SU!,GV;|v, 


NOD 3 

W11 M HEP 8 

S"ALLMi*! "(MMi.HED S‘<AU,\U1 

M"'4 

f* 0 H E p s 

SMALL.* (H "HUMPF.0 

MJi-lb 

MJ-IHrk * 

"HUWUhEO "AM) ShALLMM* 

NUhf 

KMWFP s 

"HUM HE^ smallM r' 

f i U M 7 

SMALLMJM 

s DIGIT 

MJMb 

SHAI.LWIJFi 

8 TEEM 

NbAlO 

TEEN 8 DIGIT -TEEN 

HUM 11 

PICT! s DIGIT -TV 

HUM 12 

SMALLMIP 

8 DIGTY 

NUMlJ 

SK ALLMJR 

s DIGTY DIGIT 


FIGURE 21 CONTEXT-FREE PRODUCTIONS OF BASE GRAMMAR (Concluded) 
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function SEMRSJ . if SEMRSi returns ML, tne rule fans on semantic 
grounds. Otherwise, SEMRS1 returns tne semantic interpretation of S, 
and the other attributes of S are computed in accordance with tne 
attributes statement and stored with the completed S. 


While space considerations make it impractical to 
list the complete grammar in this document (see Ref, 3f> for a 
complete listing), a feel for the extent of the grammar may be gained 
by examining just the context-free productions of the rules. The CFPs 
of the base grammar are presented in Figure 21, with CBC extensions 
in Figure 22. (A base rule marked by "coc" (or "speech") indicates 
that the rule is currently used primarily by the CBC (SRI speech) 
project.) 
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FIGURE 22 CBC GRAMMAR EXTENSIONS 
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7. 


The Parser 


The SRi speech group has developer) an experimental 
parser that is specifically d e siqned to coPe with the variability of 
pronunciations and the frequent lacx of separations between words 
that characterize spoken lanauage. To take advantage of knowledge 
about the input stream from any sources that are available, the 
parser is allowed to (simultaneously) work both top down and bottom 
up, predicting words and Phrases on the basis of context and building 
up phrases and clauses from words that have been identified in any 
position in an utterance. Usinq information from many sources, the 
parser coordinates knowiedqe relating to the structure of bnqllsn, 
the natuie of cne task, being performed, the conversation context, the 
prosodic features of the input, and so on. Since the uncertainty of 
the input and the variety of kinds of Knowledge required can lead to 
consideration of a large ranoe of interpretations in the analysis of 
an utterance, the parser contains mecnanisms that enable it to 
examine the most reasonable alternatives first and to focus its 
activities with respect to both processing time and available space. 
(A full description of the parser is contained in Ref. 36.) 


To facilitate the conversion to spoken input in 1978, 
we are designing our interim text processor to be as nearly like the 
proposed speech processor as possible. Toward this end, the speech 
parser has been modified to accept text input. Although a simpler 
parser could oe written to do text processing exclusively, by using a 














modified speech parser in the interim, the eventual conversion to 
spoken input will entail no interface changes between the parser and 
the CBC-specific systems with which the parser communicates, noth 
the CBC-specific systems that communicate with the Parser and the CBC 
grammar may be written and tested in text mode. Further, the 
text-based system will serve as a useful tool for testlna 
task-oriented input-prediction strategies that will be needed to help 
guide the parser through spoken consultation dialogs. 

8. System Design 

Since the language system must be coordinated with 
other components of the CbC, we have attempted to temper our work in 
protocol analysis and in the development of a semantic representation 
by keeping in mind the interactions that must occur between the 
language system and various knowledge specialists in the CBC. To 
better understand the intercommunications problems, we have developed 
tentative system designs at two levels. 

The most abstract design, shown in Fiqure 23. 
interprets the language component as the interface between the user 
and a group of task specialists. According to this model, the 
language component's Mediator handles interactions with the user and 
with various subsystems that are highl/ competent in the performance 
of restricted and specialized tasks. In interacting with the user 
and the task specialists, the Mediator receives help from a user 
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FIGURE 23 ABSTRACT LANGUAGE SYSTEM DESIGN 


*odel and a task model. The user model encodes the discourse 
history, the state of the user's proqress in the performance of some 
task, and Information concerning other characteristics (e.q,, 
acoustic parameters and, eventually, problem solving style) of the 
current user. The task model contains abstract descriptions of the 
task specialists' capabilities and of the tasks that the user himself 
can perform. Both the user model and the task model help the 
Mediator understand the user's utterances (principally by providing a 
context for the resolution of anaphora and ellipses). The task model 
helps the Mediator decide which task specialist to Invoke in seeking 
to respond to a user's queries. 
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At the more concrete level of software modules# we 
are working with the system design of Flqure 24 In mind. In this 
figure# double-lined arrows represent flow of control# solid arrows 
represent flow of information# and broken-line arrows represent 
subroutine calls. According to tnis system desiqn# control is first 
resident in the parser which receives data from the arammar and the 
lexicon (and# of course# from the user). lr the course of its 
operations# the Parser is expected to call on the Composition 
Semantics Subprocessor# which builds up semantic interpretations of 
parse fragments. To perform the semantic interpretation# the 
Composition Semantics may request information from the 
Pragmatics/Discourse unit or from the data base (through the Data 
Base Controller). 


Unce a semantic interpretation of a complete input 
has been ccnstructed# it is passed to the Request Server. Since the 

CBC cannot carry out physical commands# user inputs are either 

requests for information or statements of fact for the system to 

commit to memory or use in updating its model of the task being 

performed. In either case# the Request Server calls the Data Base 

Controller, which may input new data into the data base (and 

indirectly cause updating-monitoring demons to oe fired off)# or may 

query the data base (and perhaps indirectly invoke knowledge 

specialists, such as the planner# that derive pieces of data not 


specifically encoded in the data base) 
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When an appropriate response has been determined by 
tne Request Server, the response is passed either to the languaqe 
generator (which produces spoken output), to the physical generator 
(whlcn provides a mechanical output such as the pointing of the IV 
camera or the laser), or to both. 

The relationship between the software-module system 
design and the more abstract design is indicated py Figure 25. 

9, Semantic Representation 

One of the most fundamental tasks in the construction 
of any artificial lntelllaence system Is the selection and 
Implementation of a scheme for representing knowledge, within the 
past year, our research in semantics has concentrated on the 
development of a modeling scheme for use In natural language 
understanding that Is capable of encoding multiple aspects of 
knowledge In a uniform, precise, and easily manipulatable form. 
Special emphasis has been given to the uniformity of representation 
since, depending upon the point of view of a processing algorithm or 
the disposition of the apprentice, a given piece of data may be 
regarded internally or expressed in output in a variety of ways. 

The representation scheme that has oeen developed for 
use In the CBC Is a variation of the semantic network schemes used by 
Simmons 139J, Shapiro 140J, Humelhart and Norman 141J, and other 
researchers. For natural language applications, networks have proven 
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FIGURE 25 RELATIONSHIP OF ABSTRACT DESIGN TO SOFTWARE MODULES 









































to oe very serv ceatle encoding structures* ottering such recognized 
advantages as a convenient bidirectional linkage between semantically 
related data items and an inherent facility tor associating deep 
conceptual case systems with event types. To extend the encoding 
power and flexibility ot semantic networks, w e have developed an 
augmentation that allows the nodes and arcs of networks to be 
partitioned into units called spaces, This allows knowledge to be 
bundled into units* thereby helping to condense and organize the dfta 
base. Specifically* partitioning allows quantification* abstraction, 
and categorization to be handled easily by networks. Partitioning 
also facilitates the encoding of process infornatlon and the 
translation from English into network representations, (A discussion 
of partitioned semantic networks is contained in Kef, 42. 

a, Basic Network Notions 

In its simplest form, a semantic network is a 

set of nodes Interconnected py an accompanying set of arcs. A node 

maY be used to represent an object* where an object maV be virtually 

anything, including physical objects, relationships, sets, events, 

rules* and utterances. Arcs are used to represent certain 

« 

"primitive" omnichronic (l.e.* time invariant) relationships, (Such 
relationships may also be represented as nodes.) 

A feeling for how nodes and arcs are 
organized to represent various tacts may be qaine, by considering tne 
network of Figure 26. In this network the node 'PHYSICAL.OBJECTS' 












FIGURE 26 A TYFICAL NET FRAGMENT 


(single quotes aie used to delimit nodes) represents the set 
PHYSICAL.OBJECTS, the set of all physical oblects. Likewise, node 
'MACHINE,PARTS' represents the set 0 t an machine parts. The arc 
labeled "s" from 'MACHINE.PARTS' to 'PHYSICAL.OBJECTS' indicates that 
MACHINE.PARTS is a subset 0 f PHYSICAL.OBJECTS. Similarly, the 
network Indicates that BOLTS is a subset of MACH1NE.PARTS and that B, 
an Element of BULTS, is a particular bolt. Following the hierarchy of 
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anotner family# X Is a particular box, an element of BUXES, a subset 
Of CONTAINERS, a subset of PhYSICAw.CBJECTS. 

Node 'C # encodes a containing situation, an 
element of the situations set <slt-contain>, a subset of SITUATIONS, 
tne set of all situations. In particular, 'C' represents the 
containing of bolt B by box X from time T1 until time T2. The 
various component parts of situation C are associated with it through 
special deep case relationships. For example, in the network there 
is an arc labeled "content" from 'C' to 'B', Tnls arc Indicates that 
B is the #@content of situation C, where the notation "i«content of 
C" means "the value (#) of the content attribute ( a ) of C." 
Similarly, X Is the #Pcontalner of C while T1 and T2 are the 
#@start-time and #@end-tlme respectively. 

As a general principle, arcs encode only 
element, subset, and case relationships. CUnder one interpretation, 
element and subset relations may be viewed as deep cases also.) Arcs 
are never allowed to encode relationships, such as ownership, which 
are time bounded, 

b. Net Partitioning 

The central idea of net partitioning Is to 
separate the various nodes and arcs of a network Into units caned 
spaces. Every node and every arc of the overall network is assigned 




























partitioning is enclosed within a dotted line. For example* space 
SI is at the top of the tlqure and includes nodes 'PHYSICAL.OBJECTS', 
'BOLTS'* '<sit-contain>'* and 'BOXES'. Si also includes the two s 
arcs indicating that the set of BOLTS and the set of BOXES are 
subsets of the set ot PHYSICAL.OBJECTS. In our dlagramatlc 
representations of semantic nets, an arc belongs to a space if* and 
only if* the arc's label is written within the dotted line boundaries 
of the space. Thus the e arc from 'B' to 'BOLTS' lies in space S2. 

The various spaces of a partioning are 
organized into a partial ordering such as that shown in Floure 28. In 
viewing the semantic network from some point S in this ordering, only 
those nodes and arcs are visible that lie in S or in a space aoove S 
in the ordering. Thus, tor example, from space S2 of Figures 27 and 
28, or.iy those nodes and arcs lying in S2 or Si are visible. In 
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FIGURE 28 A NET-SPACE PARTIAL ORDERING 



















particular/ it is possible to see that a is a BOLT and that BOLTS are 
PHYSICAL.UBJLCIS, hut it is not Possible to see that X is a Box. From 
space S5/ information in spaces S5, S3, S2/ and si is visible. Hence, 
from S5, the whole of the semantic network of Figure 27 may oe seen. 
CFoi certain applications, the net may oe Inspected one space at a 
time. For example, it is possible to query the net in such a way that 
only nodes and arcs lying in space S2 are visible even though 
information in Si is normally visible whenever S2 is inspected.) 

in practice, partitioned networks are 
constructed by creating empty net spaces, adding them to tne 
partition ordering, and then creating nodes and arcs within each 
newly created space. The use of partitioning in the encoding of 
quantified statements and categories is the subject of the next two 
sections, 

c. Quantified Statements 

In addition to an ability to encode specific 
facts (such as the containing event encoded in Figure 2b), a semantic 
system needs some facility for grouping sets of similar facts into 
units--so as to allow those facts to be represented collectively 
through some sharing mechanism and to be conceptualized as an 
integrated whole. An ability to encode generalized information (in 
the form of quantified expressions) is of considerable importance 
since it is often impractical (or even impossible) to record the same 













information by a collection of Individual specific statements, noth 
because of the very number (possibly infinite) of statements 
required, and because details of particular individuals may not be 
explicitly known. Further, since quantification is a component of 
language, an ability to encode quantifiers is vital to the 
understanding of certain classes of Enqlish sentences. 
(Quantifierr were seldom used explicitly in the dialogs gathered in 
our protocol experiments concerning mechanical tasks. 
Nevertheless, quantification is needed for the representation of 
world knowledge.) 

As an example of how quantification is 
handled in partitioned networks, consider the network of Figure 29 
which encodes the statement 

"Every bolt in the box is 3/4 inch long 

and has a nut screwed onto it." 

In this network, the node 'CiS' represents the set of all general 
statements (the set of statements involving universal quantifiers or, 
under another interpretation, the set of recurring patterns of 
events). The node 'g' represents the particular statement (set of 
events) cited above. 

Characteristically, a general statement 
encodes a collection of separate circumstances, all of which follow 















Th# bolls 


has length 


3/4 INCH 


I_ 


<»l-*cf*»v«d 5>mplmic> 


SA-3805-20 


FIGURE 29 EVERY BOLT IN THE BOX IS 3/4-INCH LONG AND HAS A NUT SCREWED 
ONTO IT 


the same basic pattern. This basic pattern is represented by the 
Mform of the qeneral statement. The monn of g is encoded by a 
net space, S4, which lies just below SI in the partition ordering. 
(When one net space is pictured Inside another, the inner space is 
below the outer in the partition ordering.) This net space may be 
thought of as a super-node, containing its own internal structure ana 


















representing a composite variable that takes on a different value for 


each of the instantiations of the recurring pattern. Eacn node and 
arc within the space of the super-node may be thought of as a 
subvariable. 


General statements are also typically 
associated with one or more universally quantified variables, which 
are allowed to assume values in some specified range. Statement g, 
for example, has a universally quantified variable b given by its flvv 
attribute. Note that variable b is necessarily a part of the inform 
of g (i.e., 'b* lies in space S4). From node *o' there is an e arc 
to the set the.bolts.in.the.box, indicating that the value of b 
(written lb) must be taken from the range set the.bolts ,in.the.box. 
(The node 'the.bolts.in.the.box' has been created especially to help 
encode the general statement. Its meaning may be inferred 
subsequently when the,bolts,in .box,X is defined in paragraph "d" 
below, 


The interpretation of a general statement is 
that for each assignment of the variables MVv to values in their 
corresponding ranges, there exist entities matching the structure of 
the Inform, For g this means that for every lb an element of 
the,bolts,In.the,box there exist 

Ih C <has.length> 

is C <sit-screwed*simplistic> 


in C NUTS 



























and the relations 


I b is the # P o b j of #h 
3/4INCH Is the IPmeasure of I h 
»b is the #f*mt of # s (l.e., in is the male 
-threaded part of is) 
and in is the #«ft of is. 

Thus, the interpretation of q is that fo r e v e ry # b ln 
the.bolts.in.the.box, there exists a situation #h in which the lenoth 
of UBJect #b is the MEASURE 3/4 inch. Since '3/41NCH * lies outside 
space S4, there is only one measure for all the holts in the box. 
Further, tor each holt ib there is a nut m (depending on the 
individual # b ) that is in a situation of being screwed onto #b. (A 
screwedjsimplistlc connection may exist only between two threaded 
objects, one with male threads (the #Hmt) the other with female 
threads (the #@ft). A screwedjsimplistlc connection may be contrasted 
with screwed:standard connections in which multiple unthreaded parts 
are held toqether by a bolt (or other threaded object) that passes 
through the un h readed objects to enqaqe a nut.) 

Complex quantifications entailing nested 
scopes may also be encoded by net spaces, as shown abstractly in 
Fiqure 30. 
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FIGURE 30 A COMPLEX ABSTRACT QUANTIFICATION 
(VaeA)(3beB)(VceC)(ddec)[P(a,b,d)] 


d. Rules and Categories 

A convenient method for organizing 
information in a semantic system is to divide the various ODjects 
(Including physical objects, situations, and event objects) in the 
semantic domain into a number of categories. Using categories, 
objects that are somewhat ali<e become grouped together, allowing 












similar objects to be thought about and talked about collectively. 
The scheme is hierarchical in that some categories may be 
subcateqories of more general classes. rhe lower a class is in the 
category hierarchy, the more alike its members must be. The 
'likeness" arises in that members of each category possess certain 
common characterizing properties (such as deep conceptual cases or an 
association with a common attribute. 

The categorical system serves the important 
purposes of spotlightlnq similarities among objects and compressing 
redundant information by recording common information at the category 
level rather than with the individual. if an object Z is known to 
belong to some category K, then Z is known to possess the common 
properties of K's members and the common properties of the members of 
each of K's supercategories. This ability to encode information at 
the category level rather than with each inaivldual is of practical 
Importance because it saves computer memory and because all the 

elements of a category (perhaps being infinite in number) may not be 
explicitly known. 

For natural language processing, the category 
system has the important feature that members of the more significant 
categories (the categories commonly held in the minds of humans) are 
expressed by the s a me set of linguistic Patterns. As an elementary 
example, screwdrivers, wrenches, hammers, and saws belong to a 
category of objects that may be expressed by noun phrases headed by 













the noun "tool." Various attaching events may be expressed by 
complete sentences using the words "attach#" "mount#" or "fasten" as 
their central verbs. 

Central to the notion of a category Is the 
notion of a rule that specifies a necessary and sufficient test for 
category membership. Necessary rules# which all cateaory members 
must obey# and sufficient rules# which can prove that an object 
belongs to a qiven category# are also of importance. 

As a simple example of a category and Its 
defining rule# consider the category of bolts In box X. This 
category is represented by the node # tne.bolts.in.box.X' of Figure 
31# with the associated rule being encoded by net space S2. The 
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FIGURE 31 THE NECESSARY AND SUFFICIENT RULE DEFINING "THE BOLTS 
IN BOX X" 











ens arc *ying in space S2 from node 'b' to ' the . Polts.1n. pox.X # 
Indicates that ' b' represents what may dp t h o u g n t of as an archetypal 
element of the category. (Symbol "ens" means archltypal element, 
necessary and sufficient.) Any object with the characteristics ct b 
be lores to tne category, and all members of the category have the 
characteristics of b. As encoded in space S2, the characteristics 
of b include membership in ttULlS (the set of all bolts) and 
involvement as the ^content i n d containing situation in which box X 
is the #Pcontainer. 

In natural language processing, particularly 
during the parsing phase when surface structures are being translated 
into nets and when the semantic well formedness of sentences and 
sentence fragmerts is being tested, it is important to Know what 
attrlDUtes (deep cases) are associated with certain categories of 
objects (especially with event, situation, and other vero^llKe 
cateaorles) and wha* ranae of values each attribute may assume. This 
Information has utility because attributes indicate the types of 
participants that are involved in particular categories of situations 
an ‘ because there is often a direct mapping from syntactic cases 
(including prepositional phrases) to these attributes. Knowing tne 
correspondences between surface cases and attributes and knoving the 
ranges of values tor each attribute allow some oarsen to be rejected 
on macrosemantlc grounds and provide a facility for predicting the 
citing of certain situation participants in the surface language. 
(This prediction ability is especially important tor speech 
understanding.) 







ne actribute-ranqe Information tor a 
category, collectively referred to as the category's delineation, may 
be associated with the category through a delineation rule. A 
delineation rule is a necessary rule that includes ranae information 
about every attribute of the delineated category. 

As an example of a delineation rule, consider 
the delineation of category <to-bolt>, the category of events in 
wnich two machine Parts are attached bY using bolts as fasteners. 
Delineation information tor this cateqory is encoded by the network 
of Figure 32. In this network, tne node '<to-bolt>' is linked to a 
node 'b' by an ed arc which indicates that o is the delineating 
"element" of <to-bolt>. Fncoded within space S4 is attribute-range 
information concerning each of the six attributes Possessed by 
members of <co-bolt>, in particular, the rule encoded by space S4 
indicates trat a bolting event involves an factor taken from tne set 
of INTELLIGENT,ANIMATE,OBJECTS, a «aminor-p and a #@major-p taken 
from the set of MACH jNE.PARTS, a set of #Pfasteners taken from tne 
set of BOLT/NUTS ia bolt/nut is a bolt and a nut which work together 
to form a single fastener), a iptool taken from the set of TOOLS 
(which includes hands and fingers), and a l^time taken from the set 
Of TIME.INTERVALS. 


Given the two sentences 

"I bolted the pump to the base plate wlTH 1 
INCH BOLTS," 
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ens” arc lying in space S2 from node 


b 


to 'tne.bo1ts,1n.oox,X 


indicates that 'b' represents what may dp thouqnt of as an archetypal 
element of th* cateuory. (Symbol "ens" means architypal elemert, 
necessary and sufficient.) Any object with tne characteristics of b 
belorgs to the category# and all mempers of the category have the 
characteristics of b. As encoded in space S2, the characteristics 
of b include membership in PULIS (the set of ail bolts) and 
involvement as the ^content in a containing situation in which box X 
is the ^container. 


In natural lanauage processing, particularly 
durinq the parsing phase when surface structures are beinq translated 
into net:; and when the semantic well formedness of sentences and 
sentence fragmerts is beinq tested# it is important to know what 
attriDUtes (deep cases) are associated with certain categories of 
objects (especially with event, situation, and other verb-like 
categories) and what range of values each attribute may assume. This 
information has utility because attributes indicate the types of 
participants that are involved in particular categories of situations 
and because there is often a direct mapping from syntactic cases 
(including prepositional phrases) to these attributes. Knowing tne 
correspondences between surface cases and attributes and knowing the 
ranges of values for each attribute allow some parses to be rejected 
on macrosemantic grounds and proving a facility for predicting the 
citinq of certain situation participants in the surface language. 
(This prediction ability Is especially important tor speech 
understanding.) 
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single rule linked directly to the category 


In practice* 


categorical information is almost always distributed among many 
points in tne categorical hierarchy. To see how information is 
distributed at various levels* consider the hierarchy of <to-attach> 
events that are shown in Figure 33. The most aeneral category in the 
hierarchy is cateaory U* the universal set, tven U has a 
delineation* since a’l objects (including events and situations) 
exist over some (possibly one-point or infinite) time interval. A 
subset of U is <to-attach>, the set of all attaching events of any 
nature whatever. Members of <to-attach> inherit the time attrinute 
from supercategory U and add two additional attributes* departs and 
factor* of their own. In general* each attaching event involves a 
set of departs that an d@actor binds together in some way. 

Two subcategories of <to-attach> are shown in 
Figure 33, The first is <to-screw:simpllstio* which is the set of 
events in which two threaded objects, one (*@mt) with male threads, 
the other (#@ft) with female threads* are engaged by twistina. Notice 
that the delineation rule of this category shows that the #0mt and 
the d@ft are both elements of the dSdoarts, The cardinality of 
departs is at most two (but could be one* as for a garden hose with 
one end attached to the other), 

A second subcategory of <to-attacn> is 
<to-attach:fastener>* the category of fastening events in which the 
departs are attached with fasteners, (Screwing a light bulb into a 
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socket requires no fasteners and is a simplistic screwing event. 
Nailing a sign to a post requires a nail as a fastener.) The 
delineation of <to-attach:fastener> simply adds the attriDUte of 
$fasten^rs, 

Category <to-bolt> Is a subcategory of 
<to-attachitool> which is a subcategory of <to-attach: fastener>. The 
delineation of <to--bolt> shown in Figure 33 Indicates how the 
f@major-p and the #Pminor-p are related to »Pparts and to each other. 
Further, the ipfasteners used by boltina events are restricted to be 
bolt/nuts as opposed to any type of fastener. Linkage to a process 
automaton that indicates the sequence of changes characterizing a 
bolting event might also be included with the category information 
but has been omitted here for simplicity. 

e. Abstraction 

Since a user may think at varying levels of 
detail, it is important for the semantic system to be able to encode 
information at multiple levels of abstraction and have some 
capability for jumping from one level to another. figure 34 shows 
one way in which net Partitioning may be used to encode an attaching 
event A at two levels of detail. By viewing the network from the 
vantage of space S2 (which lies below St in the ordering and is a 
sister space to S3), A is seen to be an element of <to-attacn>, since 
the e arc lying in S2 is visible. Since the information lying in 
















FIGURE 34 VIEWING A BOLTING AT TWO LEVELS OF DETAIL 


S3 is Invisible from S2, A appears to have only an factor and a set 
of Imparts and Is not seen to Involve *@fasteners, From S3 the same 
event nay be viewed with more detail, First, the e arc trom A to 
<to-attach> Is invisible and A Is thus seen, as an element of 
<to-bolt>, a subset of <to-attach>. Further, at this finer level of 
detail, the ^fasteners involved In the attaching (bolting) event are 
visible (as are tools and the liKe, wnlch are omitted from the figure 
for simplicity). 
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A very Important aspect of the workstation 
domain Is that of chance. Since sequences of cnanqe tend to follow 
certain reqular patterns/ it Is convenient to orqanize the recurrlnq 
sequences of chance Into categories# grouping similar sequences 
togetner, Each category of sequential change is tantamount to an 
event category# the members of whlcn may ne thought of as Individual 
enactments of a common plot or script wnich encodes a generalized 
pattern of change. For example, every event of tightening bolts 
follows the plot consisting of finding a wrench# putting the wrench 
on the bolt# twisting the bolt clockwise# ^nd so on. bach enactment 
casts different actors in the various roles, but follows the same 
basic pattern. 

Since the members of a particular event 
category may be distinguished as exactly those instantiations of 
sequential change that follow s;ome particular script# the script 
itself forms the basis for a rule defining the event category. 

During the past year we have been considering 
ways to encode process scripts in semantic networks for use in 
language processing. The procedural nets developed by the CdC 
Planning group are, of course, a representation of process knowledge, 
and we anticipate the eventual merger of procedural and semantic 
networks- However, since procedural nets were not designed with 


























language processing In mind# we have considered Process automata 142] 
as a possible alternative. A process automaton is a section of 
semantic network that resembles a Mealy machine or a Woods AFS1N 
system [43]. While AFKTn was aeveloped as a programming structure 
to describe the process of parsing language, the process automaton 
has been developed as a data structure for descriolno the processes 
(prototypal plots) cited by language and may oe reqarded as a parsing 
grammar that interprets (or generates) a sequence of changing 
conditions rather than a sequence of words. If a Pat h can be found 
through a process automaton network for a qiven sequence of changes, 
then the sequence is accepted as a "word" (an enactment) in the 

"language" (category of events) defined by the "grammar" (process 
automaton), 

During the next year, further study win be 
given to the problems of encoding scripts in networks; a common 

ground for procedural networks and process automata is expected to be 
found, 

g » Translating into Semantic Networks 

Although no attempt has yet been made to 
process natural language in the CBC domain, the semantic translation 
routines written for the SHI speech project were developed with the 
requirements of the CBC domain in mind, since the techniques for 
building network representations of input sentences are discussed in 




















Ref. 3b the description will not oe repeated here, with the recent 
modification of the speech parser# whicn allows it to accept text 
input# semantic routines tor translation CBC related data are 
expected to be available within the next six months. 


10, Re latino the Semantic Structure to Discourse Needs 

In the section on dialog analyses we mentioned two 
problems# reference and object description# which were closely 
related and which may now be discussed further. As mentioned earlier# 
solutions to both of these problems depend on being able to supply 
contextual focus# that is# an ability to restrict the system's 
attention to a small but pertinent subset of its total Knowledge. 
Since the system's Knowledge is recorded in a semantic networK, a 
form of net partitioning may be used to group together those facts 
that are liKely to be pertinent at a given noint in the dialog. For 
tasK-oriented dialogs# the division of tne dialog into coneslve 
subdialogs is closely tied to the taSK structure, mis structure is 
embodied in the procedural net, which encodes tne tasK structure in a 
hierarchy of subtasKs. Grouping the information relevant to each 
subtasK into a separate net space# and ordering the net spaces in 
accordance with the procedural net hierarchy, produce a Knowledge 
structure that supplies contextual focus. 

Figure 35 is the semantic net representation of a 
wrench w and its relationships to other objects (by "objects" *e mean 
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WRENCHES 



FIGURE 35 SEMANTIC NET FOR WRENCH, W. WITH LOGIC PARTITIONING 



























any entity that is encoded as a node in the semantic net). Note that 
tnis is only a fragment of a semantic net: only a subset of the 
relationships in which w might participate are shown. The 
partitioning shown is the logical partitioning described in section 8 
above. Space Sw of this partitioning is used to delineate the class 
of wrenches, and indicates that each wrench has a #fasize and an 
it?endtype. (These components of a wrench's description are indicated 
through case relationships because they are time invariant, intrinsic 
properties. Neither the size nor the enatype of a wrench may ce 
altered without destroying the wrench itself. Node structures could 
have been used to encode this information, but such an encoding is 
more expensive and is not needed here.) wrench w, an element of 
WRENCHES, has itasize 1/2-inch and ifesendtype BOX-END. In addition to 
the intrinsic properties of size and endtype, wrench W has the 
distinction of having been used (as the Mtool) in the attaching of 
the pump to the platform between times Ti and l’j and of belna in 
(being the iPcontent of) the apprentice's left hand from time Tie to 
the present. (NOTE: "Time" arcs go to intervals. An entity may also 
have a #@start-time and a #Rend-time* in this case the interval is 
C»@endtime,#$starttlme)). 

All of this information is part of the history of 
wrench W, As such, any of it may oe used in the description of w. 
However^ in any qiver. contextual focus only some of it is valuable. 
For this reason we would like to be able to hlghliqht certain arcs 
and nodes in the network while they are in focus, letting them return 
to their unhighlighted state when the focus changes. 




















in order to do tnis, network partitioning is used in 
a new way. Nodes and arcs belong to botn logical and focus spaces. 
The logical and focus partitions are orthogonal to one another in the 
sense that the logical space upon which a node or arc lies neither 
determines nor depends on tne focus space in which the node or arc 

lies. (The focus partitioning differs from the logical one in several 
ways wnlcn we i»m discuss shortly). 


The Procedural net representation of a task encodes 
doth the subtask hierarchy and the partial ordering of subtask 
performance for that task. For any given execution of the task, only 
a subset of the nodes m the Procedural net are invoked. These 
correspond to the subtasks actually discussed by the apprentice and 
the expert. For example, it the expert directs the apprentice to 
attach the belt housing cover and the apprentice replies by saving 
that he has done that, then the nodes that correspond to details of 
how to perform the attaching are never Invoked. 

for each subtask entering the dialog a new focus 
space is created. The procedural net imposes a hierarchical ordering 
on these spaces. This hierarchy win be used, as the logical one is, 
to determine what nodes and arcs are visible from a given space. Note 
In particular that the arcs and nodes that belong to a space are the 
only ones immediately visible from that space. Arcs and nodes in 
spaces that are above a given space are also visible, but must be 
requested specially. Other arcs and nodes 


are not visible. 













As mentioned previously, tne focus partltionino 
differs from the logical partitioning in several ways, hirst, a node 
may appear in any number of focus spaces (but must appear in exactly 
one logical space). This happens when the same object i r used In two 
different suhtasks (e.g., the wrench of Figure 5). Fithft the same 
or different aspects of the object may be in focus in tne two 
subtgsks. It is also \ ossible for a node or arc to be in no focus 
space. This just means that the ooject is not strongly associated 
with the performance of any particular subtask. For completeness, we 
define a top-most space, called the “communal space," and a 
Dottcm-most space, called the "vista space". The communal space 
contains those relationships that are time-invariant (e.g., the tact 
that tools are typically found in toolboxes) or common to all 
contexts. The vista space is below all other spaces and hence can see 
everything in the semantic net. This is useful for determining all of 
the relationships into which an object has entered. 

Figure 36 shows the net of Figure 35 with a focus 
partitioning superimposed on the logical Dartlonina. Focus Fi views 
wrench W as a box-end thath which is being used in the operation of 
bolting the pump to the platform. Focus Fj views the same wrench as 
one that is in the apprentice's left hand. The otner information 
about the wrench (e.g., its size) is recorded in the communal space. 
All of the information is visible from the vista space. 








































The representation of an object in a focus space will 
Include only the relationships that have been mentioned in the dialoq 


concerning the correspondina subtask or that are inherent in tne 
procedural net description of the local task. The distinction 
between the verbal context and tne general world knowledge context 
that we mentioned in Sectior 4 above (Dialog Analyses) may now he 
seen. The verbal context is supplied by the information recorded in 
the subspace hierarchy. Tne general world knowledge context is 
information that is present in the communal space. ^hen resolving a 
reference^ we can decide how to divide effort between examining links 
in the local space and looking back into the communal space. 


An advantage of adding this new partitioning is that 
special information can be recorded at the local focus level. Thus if 


several links in the net must be followed to establish some fact 
about an object (i.e., some logical deduction must he done)/ the 
result of that work may oe stored explicitly in the local focus 


space. The logical deduction does not have to oe redone for local 
references. If this information is put in its own logic space/ then 
it remains invisible from the knowledge net (the topmost logic 


space). For example/ consider the situation portrayed in Figure 37. 
All of the nodes and arcs in this figure are in one focus space, FI. 
B-E is a set of box-end wrenches to which wi belongs. H-E is a set of 


hex-end wrenches to which W2 belongs. If the apprentice now says, 
",.. the box-end wrench", he means wi, The utterance level structure 
(created by parsing) for the phrase "the box-end wrench" is shown in 


Figure 38, and some amount of work must be done to establish tne 
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FIGURE 37 SEMANTIC NET SHOWING MEMBERS OF TWO SUBSETS OF THE 
SET "WRENCHES" 
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FIGURE 38 SEMANTIC NET FROM PARSE FOR "BOX-END WRENCH" 













































Prohlem Solvinn 


r 

- • 


1 • Freedom of v ovpment (r *) 

One way to Increase the power of our models of 
mechanical equipment is to include in the models a representation of 
the freedom of movement exls.lm between components of a device. 
Know] edae about freedo-r 0 f movement enables th«* system to answer the 
foilowino kinds of queries! 

- what linear freedom of movement exists between two 
specified components in a specified confiouration 
Of the device? CE.j.» "Thev are rinidiy attached", 
"Parti can mov e oniv in the + Y direcMon with resnect 
to Part?", "Parti and Part? do not Physically 
constrain each other", and so forth.) 

- If Parti is pushed in direction +X, which other 
components will move alono with Parti, and which 
will remain stationary? 

- How is Parti supportel? 


and, by extension. 



















If a specified operation Is performed on the device 


( e . q . # "Unscrew holt B " , "Slide the crankshaft In 
direction "Tarn the crankcase ups l de-dow n " "t 

will anv rarts lose their support# and hence fall 
nut of place? 

The last two queries require the aollitv to compute rnvslca] (as 
oonosed to logical) support. M thouah there are other# more 
sophisticated ways to comoute support# the "freedom of movement" 
approach to he described below off«n provides correct answers with 
relatively little computational cost. 

Knowiedoe about freedom of movement is beina 
Incorporated into our models at a low level. Besides facilitating 
the computation of physical support, this knov’ydqe can be used in 
the simulation of assemhly-disassembiy actions and in the automatic 
generation of disassembly suhaoals. In the remainder of this section 
we will discuss our representation and use of freedom of movement in 
more detail. 


a v Represent, inq f M 

Recall that a COMNFCTION is defined for each 
pair of components that are in contact in the canonical model of the 
fully assembled device. We characterize the freedom of movement 
(henceforth abbreviated "FM") restrictions that tne tw 0 components of 


[Hi] 




























the connection iirnose on each other 


when they are POSITIONED 


or 


ATTACHFD to one another, and associate this information with the 
CONNECTION. The FM restrictions imoos^d ny a CONnfctton could he 
represented in several different ways; for examDie: 

" As a list of restricted dtiectlons and/or 
rotations 

- As a set of linear constraints reiatina the 
positions of the two Darts 

- Deduced dtrertlv from a aeometric reoresentation 
of the shapes and nositions of the parts 

(say, by disDlacina one Dart and seelnq if it 
intersects the otheri. 

Other representations are Possible, lncludina hybrids of two or more 
of these. 


We have implemented an Fv model of the air 
compressor pump, usinq a simplified version of the first of the above 
representations, in which only translational restrictions and not 
rotations are taken into account. The current version of this model 
makes the sim P itfyj n q assumption fhat only six directions exist, 
correspondino to a standard set of three-dimensional axes and denoted 
+x, -X, +y, -Y, + Z, and -Z. In this model, CONNECTIONS are 
















classified tnco Tvdps# Mere th* tvpe character1zes the shape of the 
contact boundary between the two parts. The cnMNFCTION types 
represented are: 

ABUTS -- implies a flat, surface contact, t-ach 
part restricts the km of the other lr one 
direction, 

SPI imOIjKS -- a shaft of one part aoes throuah a 
hole in the oth»r. F'A is restricted in everv 
direction except alona the axis of the shaft. 

KESHFS -- one part is slid or seated into an 
indentation in the other, FM is restricted in 
all but one direction. 

SCPF.WS-TN -- One Part screws into the other. 

F'M restrictions depend on the state of the screw 
CONNECTION (tiaht, loose, lust positioned, or 
removed 1. 


This information is stored with the CONNECTION as the value of the 
QLISP property "TYPF." These types were chosen for their intuitive 
appeal, based on Kinds of connections that actually occur in the 
pump. Of course, any number of additional types can he defined, but 
we would like to qet by with as few as possible in order to koep the 


model simple and aeneral. we have found the above tour types, used 
















slnqly or in combination, sufficient to represent the connections 
occurrina in the oump-- a reasonabiv complicated Piece of equipment, 
as indicated in Fioure 40. 

with each connection is Stored information 
oertaininq to the orientation of the contactinq surfaces. For 
examnie, suppose three ahuttlnq Parts A, b, a nd r a re lined up alonq 

the x axis as in Fioure 41. We have two CONNFCTIdns of type abuts_ 

(A,B) and (B,C). The directional information is stored svmmetr lea i lv 
so that if we are interested in movinq part A, we discover that it is 
restricted by part B In direction ♦X• whereas if we want to move part 

B * We find that lt ls res trlcted by part A m direction -X (similarly 
for the CONNECTION between B and C. This Information ls stored 
redundantly on the CONNECTION'S Property list under the indicator 
DIRECTIONS; for example, (... DJRFC1JONS (A + x R -X) ...). as Iona 
as movement ls restricted t 0 translations, FN restrictions are 
transitive and can be viewed as rropaqatino through any rioid part. 
Now suppose that w e push o a rt A in direction fX. w e find that p a rt R 
restricts part A's FM in direction *X; likewise, nart C restricts 
Part B s FN, also in direction +X, Ip effect, the restriction 
"propaaates" from part C throuoh Part r to nart A, so that Dart C 
also restricts part A in +x. 

')f course, this simple an approach to 
represent 1 nq movement restrictions win not always work perfectly. 
For example, suppose parts a, R, and r are arranoed as in Fioure 47. 
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FIGURE 41 ABUTTING PARTS 



-— +x 
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FIGURE 42 ABUTTING PARTS NOT SQUARELY ALIGNED 


Here C does not really restrict A's movement in the +X direction. 
M*nv parts of the compressor are lined ud souarelv on#* against 
anotherr so that this kind of problem often does not arise and the 
simple model suffices. w e intend to add to the model some notion of 
shape and position so that such cases ca>" be recoarized and handled 
correctly. The semantics of assembly/disassembly oroblems suocrests 
that parts are usually held or fastened In sopcifjc "canonical" 
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confiourat i ons, and subassemblies are set down in stable positions 
during tbe assetnb 1 v/di sassembl v Process. Thus it Maht dp iruipri 
that tbe model need not represent "lumbled" confiourations suco as 
would result from pushing Part ft to the rlnht in Finure 4?. It 
would, however, bt desirable for the system to recognize when such 
"jumbled" confiourations might occur, and warn the apprentice 
accordinaly. 


Besides TYPE and DI PKCT IONS, some CONNECT T QMS 
have tbe 01 ISP pronertv STICKY. ft true value of this property 
indicates that there is enough lateral friction in the connection to 
appreciably restrict freedom of movement. 


Osina FY in 'ode'ini and Planning 


Sayino that "part R restricts part A's 
freedom of movement in direction +X" is equivalent to saying "if v 0 u 
push part A in direction + x you are also pushing part P in the same 
direction". Ignoring for the moment the effects of gravity and 
friction, we can model the effects of pushing part A in direction ♦ X 
as follows. 


we find all the parts that immediately 
restrict A's freedom of movement in direction +X* in other words, 
parts that have CONNECTIONS with A that impose the given restriction 
on A's FM. Then, for each such part we find any other parts mat 




.*■ .** --- . 
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immediately restrict its V* in the +X direction: *> continue findlnn 
all the parts so influenced (keeping track of which parts we have 
already processed so that we do not net caunht In an infinite loop). 
The final list of carts contains all those parts that also net pushed 
in the +X direction when wp push on part A. 

Tor each part P found by the above alqorlthm, 
the system remembers the oath of C0NNFCTinMs originating from part A 
^nd leading to the njven part P. This nath represents a kind of 
cause and effect chain that "explains" why part » has to move when w e 
move Part A. if an y link in this chain can oe broken (by alterinn 
one of the individual CONNECTIONs in such a way that the alven FM 
restriction is no lonaer imposed bv tnat CONK FT TT UN -- e.o., hy 
removing a fastener), then *e may be able to break the Chain of 
causality Itself and hold cart P (and all narts further down the 
Chain) stationary while we rr 0 v e Part A (and all the Parts between A 
and p). this type of analysis can generate subqoals for the removal 
and installation of Darts without having to store such suhgoals 
explicitly. For example, suppose the main ooal to he achieved is to 
remove the crankshaft trorr the pump. m order to remove the 
crankshaft, it must of course have freedom of movement in some 
direction with respect to the rest of the pumo. w e may guerv the 
model to see if such freedom 0 f movement already exists. If not, the 
model car return a summary of which chains of Darts and CONNECTIONS 
are responsible for restricting the crankshaft's movement. w e can 
then rrorose as subooals the removal of thes» restrictions, usually 
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by unfastening and/or removinq intermediate marts. Plans aenerated 
in this fashion may entail moving whole "chunks" of parts at one*, 
since the planner first achieves freedom of movement for some mart 
and then simply moves the part in the desired direction f.Iona with 
any other Parts that may he Pushed atom with it. 

c. The MOVE Function 

The function ( M OVF oarti dir part?llst) 
computes the effects of moving part parti in direction dir while 
holding the parts on oart?list stationary. The actual workinas of 
this function are somewhat more complicated than is indicated above. 
It takes Into account that If one nart supports another* then the 
supported Part win tend to follow along when the supporting Part is 
moved in any direction, unless the supported part is explicitly held 
back. (In the latter case the part may become unsupported, a 
condition that the system win detect.) Also, some restrictions on Fv 
are softer than others, in the sense that they can be overcome bv 
applying sufficient force to the parts involved. An example of a 
soft restriction is a force fit, or one Part stickina to another. 

Suppose that the + X direction in the above 
example corresponds to "up" in the real world (in other words, we 
introduce a force of aravltv into our model that Pulls all Parts in 
the -X direction). Then the chains of FV restrictions discussed above 
may Indicate which parts are supported by part A. It is true that 



















restricts movement in the up direction" is not always svnony^ous 
with "is sunportel by" sine* the "suppnrtim" part must In some cases 
be certered under the other rart to truly support It. The current 
model does not Include anv knowledge about whether oarts are centered 
In this wav. ft some cost in additional commit at 1ona1 complexity, we 
can add such information in svmnolic form, or nernans cio as far as 
derlvina it from a three-dimensional model of shapes and forces [44], 


The current model recognizes (but makes no 
attempt to analyze) the case o' an obieef supported lointlv hv 
several others. However, in the case of a single chain of FM 
restrictions leadlno from 3 ojv e n oh1e C t to a "support ohlect" (su C h 
as a table or the apprentice's hand) we can treat this chain as the 
supporting chain for the oblect. 


F.ssential lv, the «nVF algorithm works by 
"Drobma" one 0 r more parts in a given direction a n d tracing through 
the chains of hard and soft FM restrictions. By probing the part 
that is to be moved in the direction indicated, the system finds 
other parts that must move as well, Bv nrobino the parts that are to 
remain stationary in the opposite direction, it finds parts that must 
remain stationary. Then these sets of Parts are probed upwards to 
find probable chains of support, and warn of possible loss of support 
that may result from the move. 
















If the system firds that the desired move is 


imDossibie due to some chain of hard FM restrictions, the following 
procedure is followed. 

Each local F^ restriction making up this 
chair is classified accordion to the local pr econdl11 on(s) that must 
be achieved in order to "undo" the restriction. The particular 
preconditions that apply are determined from the type of CONUFCT HIM, 
the fastenlnn If any, and the direction in * hico the parts are Pelna 
probed. Table 5 summarizes these preconditions in the estimated 
order of increasing difficulty to achieve them. 

when two parts are found to have an FM 
restriction preventing a desired move, local subgoals for hreaking 
the chain are derived and ranked as indicated in the table. The 
present pump model is not intearated into a planner (the ranked 
suhgoals are printed out on the terminal): but the application to 
Planning is evident, and we hoce eventually to incorporate these 
techniques irto the NOAH system (see Section 111. B, 2). 

d. The RESTRICTIONS Function 

The function (RESTRICTIONS parti nart2) 
computes an the FM restrictions imposed on parti with respect to 
part2. (RESTRICTIONS parti Dart2) is equal to (JPPnsiTF (RESTRICTIONS 
part2 parti)). The answer is returned as a list of restricted 
directions, a subset of (+x -X + Y -y +z -Z). restrictions could be 





Table 5: Preconditions Peoulred to Undo Restrictions 


Probe 

Direction* 


Precondition to 

Type of CONNECT T O'J Undo Restriction 


Away 

Restriction imoosed 

bv a fast?nino 

1 - - S c r e w out fastener 

Awav 

CONNECTION 0 f tyoe 

SCREWS-JN 

?--Screw out the screwahie part 

Away 

MESHES or SPINDLES 

3--v 0 ve the parts In the direction 

Permitted bv MESH or SPINDLE 

Toward 

ABUTS 

4--M 0 ve one part lateranv with 

respect to the other. (The best 

direction is not determined.1 

Toward 

Fastened or MESHES 

or SPINDLES or 

scpfwed-tn 

5--Actlon 1 , ?, or 3 as 

appropriate, followed bv 

action 4. 


♦Away means that the parts oniv need to be disenoaoed or unfastened 
to be free to move in the desired direction. Toward means that the 
Darts would physically obstruct each other if moved in the desired 
direction. 




























computed by probina rartl In each of the six directions and seelno if 
It moves part 7 , as in MOVE above. However, this approach, aside from 
beina inefficient, does not. oeneralize to a representation in which 
arbitrary directions are allowed. 

Another possible approach to comoufinn 
RESTRICTIONS would be to find all paths of CONNECTIONS between parti 
and rart2, and to rrooaoate all restrictions thromh each of these 
paths, finally takina the union of the restrictions imposed hv all 
such paths to arrive at the answer. This method does aeneralize to 
the case of allowlna arbitrary directions (assuminn that there is a 
method of oropaqatlnu restrictions throunh a particular CONf’ECT 1 UN) • 
but it can run into a combinatorial explosion of the number of oaths 
in a complicated device. 

The alaorlthm actually used to compute 
RESTRICTIONS is a variant of the method of Comnutlno all oaths, the 
second of the above methods. The alaorlthm takes the form of a 
search throuqh a nrarh structure in which The nodes are parts and the 
edaes CONNECTIONS. The search fans out from parti and part? 
simultaneously; FM restrictions on oartt and part? are oropaqated to 
other nodes as the search proceeds. The node chosen for expansion at 
each step is the one that imposes the most restrictions on its 
"source" part (either parti or Dart?), whenever the search fanninq 
out from Parti and Part? meets at some intermediate node, a Path is 
completed from parti to part? and new restrictions on those parts may 














be imposed. The search is terminated immediately as soon as It is 
found that Parti and rart2 are riqid ly im<ed —i.a., have no freedom 
of movement with respect to each other. This search aloorithm 
remembers the restrictions imposed by a particular connection (so 
they do not have to be computed more than once), allows those 
restrictions to he "used up" as the search Progresses throuqh a nlven 
CONNECTION, and deletes the connection from the search when Its 
restrictions are completely "exhausted". 

Stl 1 ] another method of compntlna FM 
restrictions that v.e have not implemented, hut which we clan to 

invest 1 date further, entails reoresentino the F,v between two parts as 
a bounded linear constraint on their coordinates. Then a standard 
linear nroqrammina alaorithm mioht be used to find the constraints on 
the positions of a particular pair of parts. 

e. An FxamPle 


We *111 now qive a brief example of a dialoo 
between the user and a system that models the air compressor pump. 
For the Purposes of this example, the model's attention is confined 
to a subassembly of the pump conslstino of the crankcase, head 
gasket, head, head cover, filter, filter-screen, .long head holts, 
short head holts, and head-bolt spacers, as shown m Fjqure 41. l n 
this example, minor parts such as the bolts, gaskets, and spacers are 
represented as individual parts in their own riant, and enter into 
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FIGURE 43 A SUBASSEMBLY OF THE PUMP UNIT 
































the FM computations as such. however, 
Ignore these narts and represent their 


be 

made 
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terms 

o f 

• •. 

f.'.q 


fasteninos, 

Thp function REGTN resets the model to the 
fuuv-assenbled ronfiauration and ones into an Interactive mode ln 
which it accepts mode 1-channi no commands and queries. The <,vstam 
uses the QLISP context mechanism. a r d so is generally capable of 



representino a tree of various hypothetical world states. For 
Simplicity, HFGTM hides this feature bv malntainino a "current" 
context, nushino this context whenever the state of the model is 
chanqed, and answering queries with resoect to the current state. in 
the transcript that follows, the user's inputs are in upper case and 
always tollow prompt characters (- or -»!? the system's replies are 
usually in lower case and at th- left margin? explanatory comments 
that were added for the purposes of this discussion are indented and 
enclosed in brackets r ]. 

-INITP(IMP) 

(model initialized) (Initializes the components and CONNECTIONS of 

the pump. ] 

•-BEGIN) 

(state of pump reset! 

enter commands: 

-» RESTRICTIONS HEAP-COVER CRANKCASE 

(-X +X -Y +Y -Z +Z1 (They are rigidly attached] 
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->■ SCREW-OUT HEAD-BOLTS/LONG 
ok 

-» restrictions HEAD-COVER CRANKCASE 

(+x -x -y +z -z) 

-> SCREW-OUT HEAD-POT TS/SHORT 
ok 

-» restrictions hfad-cover crankcase 
(-Y) 

-» tighten head-pot,ts/long 

ok 

-» riGHTF.H HFAD-BOI TS/SHOPT 
ok 

-» MOVE FILTER + Y CRANKCASE 


1 


A 




1 


subaoals derived from Dath: 

((? screw-out HFAD-ROLTS/LnNG) c«i nove HEAD-COVFP (-Y) HFAD-BOLTS/LONG , 
move HEAD-COVFP f+X -X +Z -7) HEAD-BOLTS/LONG) (4 move FILTER 
(+X -X +7. -Z) HEAD-COVER)) 

Freedom of movement does not exist 

[now the user plays the nart of the Dlanner and 
chooses an aoproprlate subgoai, The number at the 
beginning of each subooal Indicates its relative 
di ficulty to achieve according to the table in 
section (c.) above. The first alternative, headed 
by is derived from the CONNECTION between the 

HEAD-BOLTS/LONG and the CRANKCASE. The next, header 
hv "5”, comes from (CONNECTION HEAD-BOLTS/LONG 


w " S' 
">■ 


.-vvvw * „- v. 
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HEAP-CnvFP) . The lest, headed by "4", romes from 
( r ONNFCT T ri N FILTf.H HE A 0-CO VER 1 . J 
-» SCREW-OUT H E A D - B U L T S / LOfgG 

TThp user chooses the first subuoal ...1 
ok 

-* ML1VF FILTER t Y CRANKCASE r... and tries again.1 
tormina new chunk of moved parts consisting of 
C HEAD-BOLTS/L0MG HFAD-COVfR FILTER FTLIFR-SCREEN 1 

[The fi 11er-screen gets carried aiom because it 
sticks to the filter.] 

enter list of other moved Darts that form new connections with TABLE: 
(HFAD-cnvFR) fThe model lacks the geometric information to 

flanre this out for ltseif.1 
tormina new chunk of stationary parts consisting of 
fCRANKCASE HEAD HEAD-PPTjTS/SHOPT HEAD-GASKFT HEAD-SPACERS TABLE) 
new loose ends are* 

((CONNECTION FT1TER HEAD) (CONNECTION FILTER HEAD-SPACERS) (CONNECTION 
HEAD Hi’ AD-COVEp) (CONNECTION HEAD-COVFR HEAD-SPACERS) (CONNECTION 
CRANKCASE HFAD-BOLTS/LONG) (CONNECTION HEAP HEAD-BOLTS/LONG) (CONNECTION 
HEAD-BOLTS/LONG HEAP-SPACERS) (CONNECTION HFAD-BOLTS/LONG HEAD-GASKET) 
(CONNECTION FILTFP-SCREFN WRAP ) ) 

warnina -- the following Darts may become unsupported: 

( HE. AD-BOLTS/LONG FIlTFR-SCREEN Hf'AD-CO VFR ) 

Ok 


-> STOP 
















f. Continuing Work 

The simple Dump model described above n a s 
served as a test of freedom-of-movemenf as a modeling fool. We are 
working to develop a better understand!no of the semantics of freedom 
of tTiovemenf and mope general representations of FV restrictions. 
These concerts will be incoroorated in the model of the compressor 
and in models of other devices that may he dealt with in the CPC 
project. 

■Je beginning to examine the relationships 
between the aeometric device models bein'! used for vision and disolav 
(see Section II.C) and th* more symbolic models used for 
assembly-disassembly planning, with an eve toward automating as much 
as possible, the creation of similar reresent ations for new devices. 
Perhaps the user can characterize a new device nv the nature of the 
CONNECTIONS between its parts (as in the pump model described above) 
and the qenerai shapes of the parts, ustnq some visual Input where 
necessary. If the system had a tnree-dlmensionai shape model (even a 
crude one) and a cl assification of inter-Dart CONNECTIONS (ABUTS, 
SPINDLES, and the like), jt cool a derive freedom of movement and 
access information. This in turn could vieid planner preconditions 
for assemh]y and disassembly actions, 

T^ese investiqations raise manV of the issues 
Drevalent in currert A.T. research relatlna to "world knowledge" 
















now lt is acquired, how to renresent it internally, how to retrieve, 
onanize and undate It. In the C PC domain, we win esoeciallv 
confront the need to renresent the same device from different notnts 
of view for different purposes such as display, vision, 
assert,hiy/dlsasse-rbly Plannlna, and natural lanauaae interaction. 
Particular solutions to these Problems In the CRC domain nay shed 
some lloht on the lamer Issues Involve!. 

2» Diagnosis of Equipment 

a. background 

The use of computers to help diagnose 

malfunctions In equipment and to help dlaonose diseases In humans 
already has an extensive nlstory. Notable examples are system; that 
diagnose blood Infections ( 14 ), systems that locate faults In 

d1q11a 1 circuits [45], and systems that locate faults In Internal 

combustion engines 146 ], 

Several approaches to computer diagnosis have 
been stu u ied, and w e snail briefly mention three of them here. Twj 
approaches, using "truth tables" and using "decision trees," have 
Parallels in pattern recognition technology. A third, the 

"rule-based" approach, stems from heuristic programming techniques in 
Artificial Intelligence. 
















The so-called truth table method entails 
collecting a number of simultaneous measurements from the system 
being diaqnosed. We might represent these measurements Dy an 
n-dimensionai vector X# where n is the number of measurements made. 
Then an analysis is made of the effects on tne vector X of various 
faults that might occur in the system. In the simplest case each 
possible fault has a unique effect on X. For example# a perfectly 
functioning system might give rise to the vector Xo. Each different 
fault qives rise to a distinct vector X* for example# the 1-th fault 
dives rise to vector Xi, This association Detween faults and vectors 
can be stored in a table. In diagnosis# one enters the table uslnq 
tne measurements X to find the associated fault. 

In more complex cases the mapping between 
faults and measurements is not one-to-one. This presents a 
difficulty when the same measurement vector X could have been caused 
by a number of different faults. In the latter case# common in 
medicine# it is customary to collect a large amount of data in order 
to estimate the probabilities p(X/fauit) and then to use Bayes' rule 
to calculate the a posteriori probabilities p(fault/X). Standard 
decision theory techniques can then oe used to decide on the fault 
given any measurement vector X. 

The decision tree approach locates the fault 
by using measurements in -.equence instead of simultaneously. It is 
like the game of "twenty qv- tions." Depending on the result of the 
first measurement# another measurement is selected. Its result in 














turn determines another measurement# and so on. The alternative 
measurements can be Pictured as a tree. The identities of the faults 
are located at the tips of the tree# i.e.# after the final 
measurements in each sequence. Un the average# this approach 
requires fewer measurements than the truth taDle approach, but like 
all sequential procedures has little tolerance for errors in 
measurement. 

Several problems with these "classical" 
methods have motivated a search for some alternative approach to 
diagnosis. A n particular# we miqht mention the following: 

(1) Data qathering. The data needed for the 
truth table method are very often 
statistical. Large amounts must be 
gathered under carefully controlled 
conditions. Much more easy to gather are 
tne "distilled"# "judgemental" data used 
by skilled diagnosticians. Often chis 
information is in the form of simple 
rules like: "if symptom X is present# 
then the trouble is likely to be Y." 

An argument can be made that the 
diagnostician himself uses rules of this 
sort in his own information processing 
procedures leading to diagnosis# and# 







in particular, does not explicitly use 
tables or Bayes' rule. Presumably it 
would be more effective and efficient to 
extract tnese useful rules by intervlewina 
sxilled practitioners rather than to 
gather raw statistical data. 

Extendibi1ity* Statistical data are not 
easily extendible to slightly different 
problem areas or to situations in which 
new measurements are discovered to be 
useful. Each new problem area or advance 
in technigue would have to be accompanied 
by another extensive statistical analysis. 
Decision trees are similarly hard to 
modify to Include a new measurement. 
Generally, the whole iree must be 
redesigned, we would like a technigue 
that can easily grow to include new 
Information, 

Interaction. The classical methods do 
not allow a human expert to interact 
with the diagnostic process in any 
flexible way. In the truth table 
method, once the measurements are input, 
the process takes complete control 



























until a decision is announced. A 
human user cannot Interrupt it to 
contribute any additional information 
that might be relevant. With decision 
trees, the human user's role, if any, 
is limited to answering questions about 
the results of any of the measurements 
tnat are not made automatically, The 
human cannot volunteer any new information 
but must patiently wait until and it it 
is requested, we desire a system tnat 
can be interrupted w h en the user thinks 
he has important additional information. 


Transparency. The classical methods 
arrive at decisions tnat cannot always 
be explained to the satisfaction of a 
skilled user. This is because the user 
is likely to think in terms of judgemental 
concepts or rules as described earlier. 
When queried about a decision, all the 
classical methods can do is state 
obscure statistical facts or present 
details about the structure of underlying 
tables or trees. We want a system that 


"thinks" like a practitioner so that it 
























can explain its cnain ot deductions in 


terms understanJanie to a practitioner. 

In order to meet these criteria, we have been 
thinking about designs stemming from Al research on rule-based 
systems. A good example of such a system (even tho'gh it does not 
meet criterion 3 above) is the MYCIN medical diagnosis system of 
Shortllcfe [14], Our tentative design outline, to be described in 
the next section, is similar in spirit to MYCIN, although the 
mechanisms of operation are different and it will be an interactive 
system. 


b. System Design 

Aithougn our work on diagnosis is still in an 
early stage, we have developed a tentative design and have done some 
initial programming to test it. 

The diagnosis process begins with a statement 
of a "presenting symptom" from the apprentice. Based on this 
information a list of hypotheses aoout probable faults that might be 
causing the symptom is generated. A probability is associated with 
each of these faults. 


The fault hypotheses are generated by using 


rules of the form 


"If the symptom is X, the fault is likely 






















(probability = p) to be Y." Actually each rule is a small program 
that can perform any necessary computations required to set up the 
hypotheses. Our idea is that tne rules will be written In close 
collaborator with skilled diaqnosticians. 

In diagnosing a larqe system there are many 
levels of symptoms and faults. Suppose, tor example, that we are 

diaqnosinq a malfunctioninq automobile. A symptom miqht be that the 
enqine does not start. A "high-level - explanation miqht be that the 
iqnition system is faulty. At a somewhat lower level of the 
hierarchy, the symptom miqht be that the Iqnition system does not 
work, and the fault miqnt be that the sp a rk is too weak. Continulnq 
down the hierarchy, we might finally learn that the spark plugs are 
dirty. 

Our diagnosis system is being deslqned at the 
start to work on a hierarchical basis. Each symptom is presumed to 
be caused by faults of an appropriate level, which in turn describe 
symptoms caused by lower level faults. 

After a list of fault hypotheses is 
generated, one of them is chosen as worthy of testing. Currently, we 
are selecting the most Drobable hypothesis, but w e are oper to usinq 
a mere sophisticated criterion. After a hypothesis is selected, we 
refer to another set of rules telling us how to test for it. These 
rules are called "handlers." They are of the form: "If Y is a 
suspected fault, use test T, and, depending on the outcome, modify 

















the probability of Y." Again# the rules are small proarams that can 
perform any computations needed to help specify the test and 
interpret its results. A choice between competing handlers tor the 
same hypothesis can De made on the basis of oenefit/cost ratios of 
the associated tests. The result of applying a handler is to modify 
the probabilities of tne hypotheses. (A test may give information 
about hypotheses other than the one for which it was chosen; we will 
use "demons" to catch such additional information.) we now loop pack 
throuah the main program# select the now most-orooable hypothesis, 
select a handler for it, and so on. A trap condition allows us to 
exit whenever the probability of one of the hypotheses exceeds some 
threshold. This hypothesis is then decided upon as the fault. 

Depending on the apprentice's skills and 
desires, this fault decision may or may not be the final answer to 
his diagnostic problem. If he needs more detailed information, he 
may re-enter the fault just computed by the program into the system 
as a symptom. Tne process then uses a set of finer-grained rules to 
give a more detailed explanation of the fault. 

rte envision the system working in the 
hierarchical fashion just described, punctuated by outputs to the 
apprentice and restarting. Even so, there may be . reasons why the 
system might want to descend a few levels on its own to help make 
more definite conclusions about Mqher level faults. That is, before 
deciding definitely that a higher level hypothesis is correct, the 














system may on its own test some of the subhyootheses explaining the 

higher level one as a way of confirming or refuting the higher level 
one, 

We have written an initial UL*SP proararr to 
test some of these ideas and have applied it to problems, of 

diagnosing some simple electric circuits. After we finally decide on 
a more complex piece of equipment to reDlace the air compressor, we 
Intend to implement a much larger diagnosis system for that 
equipment. We also still must face several important questions 

involving multiple faults and intermittent faults. we win ds 
postponing the matter of how to repair faults until we make 

substantially more progress on the diagnosis problem. 

Our description of the operation of the 

hypothesis handlers glossed over the question of how probabilities 

are to be modified as a result of the tests. This is an important 
and no..trivial problem. w e nave adopted a Bayesian approach t 0 this 
matter that is discussed in some detail in the next subsection. 

c * Alternative Forms of Bayes' Rule for 

Didgnos1s 

U Introductory Remarks 

The use of Bayesian decision theory 
Is popular in medical diagnosis 147,48,49), and has a natural appeal 















In any diagnosis problem in which one has suagestive but not 
compelling evidence to support various hypotheses. As Shortliffe has 
pointed out 150], direct use of Bayes' theorem is not without serious 
drawbacks. These problems are particularly severe when there is no 
cause-and-etfect model, and when correlations between the joint 
occurrence of a set of evidence and the nypotheses must be estimated 
from data. Shortliffe and Buchanan C51] give additional arguments 
for the inappropriateness of Bayes' theorem in rule-based diagnosis 
systems, and advocate their version of confirmation theory that 
substitutes certainty factors for conditional probabilities. 

This subsection presents several 
variants of Bayes' theorem, some of which are very similar to 
Shortliffe's certainty factor formulas. The variants are appropriate 
under different circumstances, We begin with the simplest case of a 
single hypothesis and binary-valued evidence, 

11) Bayes' Rule for Events 

Let H be a hypothesis that is either 
true or false, and let E be a oiece of evidence that either is or is 
not present. We view E and H as random events, writing P(E) as the a 
priori probability of the occurrence of event E, P(H) as the a priori 
probability of event H, and P(E,H) as the joint probability of E and 
H, l.e., the probability of the event EAh. For the nonoccurrence of 


E or H, we write 















Q(E) = P(E) = l-P(E) 


(Cl 


and 


Q(H) - I’(H) = l-P(H) . (CJJ) 

For the joint probability function, w e -r,ust know three of the tour 
probabilities P(E,H), P(E,H). P(E,H) and P(fT,H). By the law of total 
probability, P(E) and P(H) can oe derived from the complete joint 
probability function by 


p (E) = P(E,H) + P(E,P.) (Ci) 

p (H) = P(E,H) + P(E, H ) . 

(C 4) 

The conditional probability P(E\Hj is 

defined as 


p(e|h) = 

P(H) 


(C 5) 


Similarly, 


















with analogous formulas for P(L\H) and P(H\b). now, typically, we 
either know or can estimate both P(tl\H) and P(f\h), we obtain the 
evidence t, and we wish t.o update the prooability of h from the a 
priori value P(H) to the a posteriori val le P(H\E). Bayes' rule, 
which follows immediately from the above formulas, provides the 
desired answer: 


P(H |fi) = _ 

P(E H)P(H) + P(e[h)P(H) 


(C 7 ) 


Hi) Some Useful Variants 


Define the likelihood ratio by 


A(E|H) 


P (E 

H) 

P (E 

H) 


CC8) 


and let the ratio 0(H) be the odds favoring hypothesis H, 


0(H) 


P(H) _ LOil 
P(H) Q(H) 


(C9! 


Then we can rewrite tq. tC7) as 


P(h|e) = 


AO 

AO + 1 


(cio: 










Thus, the a Posteriori probability can be determined from tne 
likelihood ratio and the a priori odds, Alternatively, if we use the 
fact that 


e(h|e> = 1 - p(h|e) = 


and define the a posteriori odds by 


o(h|e) = 


P (H 

El 

P (H 

E) 


(cm 


then we obtain the simple formula 


0<h|e) - A(E|H) 0(H) (Cl 2) 

This formula is sometimes called the 
odds-ilkelihood formulation of Bayes' rule 152J . It shows how the 
oids favoring a hypothesis change when new evidence is obtained. For 
our purposes, it Is perhaps the most useful form of Bayes' rule. It 
suqgests a simple recursive updating procedure that win be 
elaborated in Subsection v oelow. To recover the probabilities from 
the odds, we merely use the formulas 


P(H) = 


0(H) 

0(H) + 1 


(Cl j) 
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and 


P (H | E ) = 


0(H IE) 
0(H|E) + 1 


(C 1 4 ) 


iv) Objections and Problems 

There are two basic problems in uslnq 
tnese results. The traditional objection concerns the a priori 
probabilities. If H is the hypothesis of an event that has never 
been known to occur naturally* then even thouqh we may be able to 
force H to occur and thus determine P(E\H), there may not be any 
objective w a y to determine P(H), Although this can be viewed as a 
philosophical objection to the whole approach, the degree of 
seriousness of the problem has often been overstated. Even if h is a 
rare event, so that P(H) is small (but not zero), P(H\E) can be 
slqnltlcantly large if the evidence E is sufficiently strongly linked 
to H. In particular, we see from Eq. (10) that P(H\E) will be near 
unity if AO is much greater than unity, i.e., if A >> 1/P(H) when 
P(H) is small. 


Tnis leads Is to a second and more 
serious problem. It is obvious that the evidence is effective in 


confirming h it and only if it Is much more likely to be obtained 
when H is true than when H is not true. Unfortunately, while it may 
be Possible to get a very reasonable estimate tor P(FAH), it is often 

















very nard to estimate P(E\H). Consider, for example, the case in 
which H is the hypothesis that a particular car is out ot gasoline, 
and the evidence fc: is that the car win not start. Clearly, P(tAH) 
Is very near unity, but what is P(E\H)? The problem is that h is not 
a simple hypothesis put a compound hypothesis. There are many 
possible reasons for the car not startina. iwnlle w e may be able to 
estimate the probability of E given any one of them, we may find it 
hard to account for their interactions, and w e may not even pe able 
to think of all of the possible explanations. 

The oroblem we face here is not 
unlike the problem that Shortliffe faces in assigning confidence 
values to his diagnosis rules. Suppose that w e have a rule such as 
"It the car win not start, tnen there is sugiestive evidence that 
the car is out of gas." Should we assign tnis rule a confidence of 
0.1 (weakly suggestive)? 0,5 (suggestive)? 0.8 (strongly 
suggestive)? Shortliffe relies on the expert to make this decision. 
Presumably, the expert has encountered many cars that will not start, 
and has a rather aood idea of what can be concluded. 

When necessary, we adopt a similar 
attitude toward the likelihood ratio. Even thouqh the expert may be 
hard pressed to give good numerical estimates for either P(E\H) or 
P(E\H), we boldly assume that the expert can auantify subjective 
feelings about the strength of the relation between a hypothesis and 
some evidence by nivlng reasonable numerical estimates for the 






















loq-1ikellhood ratio 


l(e|h) = i ogb A(E|H) . (C1S) 

The log-likelihood ratio is of 
interest tor several reasons. It has been found to be advantageous 
in at least some experiments in which people are asked to estimate 
likelihood ratios C 5 3 J. Mathematically# the use of logarithms turns 
tne odds-likellhood formulation of Bayes' rule into an additive form 

log O(h)e) = L(EjH) + log 0(H) (C16) 

Thus, L(E\H) is an additive measure of the "information" that t gives 
about h. if l, is positive, rt is more likely; if L is negative, h Is 
less likely; it the evidence is Indifferent, L = 0, The actual 
numerical value of L depends on the choice of the base for the 
logarithm, in most of our work we have used a base, b, of 1,5, with 
this arbitrary choice, the range from -10 to MO for L corresponds to 
a range of about 1/60 to 60 for A. 

v) Sequentially Acquired Evidence 

Suppose that we have previously 
obtained n - 1 pieces of evidence El, En-1 and that we know 

t*(H\El, ,,,, Ln-1)# or, equivalently# tne odds 0(M\E1# En-l). we 

can treat these as prior probabilities (or odds) when a new piece oi 
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evidence E n is obtained, 1 h u s, we can rewrite Eg. (C12) as 


°(H l E j> •••, E n ) =A(E n |H,E 1 , . .., E n _ 1 )0(H |e^ ..., E^) 


CC17) 


where 






..., E n l) 

P(E J B ' E 1 ' E n l ) 


(Cl 8) 


This shows how 0(M\E1, ...» tn-1) can be updated recursively to yield 
u(H\Fl, ...» En), Alternatively, we can veep all n pieces of 
evidence grouped together and rewrite Eg, (C12) as 


(KhIe^ ..., E n ) = A(E^ 


.... E |H)0(H) 
n 


(c 19) 


where 



E n lH) 


P(Ej, ..., E |H) 

P(E i, •••, eJh) 


(C 2 0) 


However we write these expressions, 
one fact always remalns--the livelihood ratio depends on all n 
Pieces of evidence. In that idyllic world in which the pieces of 
evidence are conditionally Independent, so that P(En\H, ti, ,,,, 
En-1) = P(En\H) and P(Ln\H, El, ,, ff En-1) = P(En\H), we obtain 

OCHlE^ •••, E n ) - A(E c |H)0(H|E 1 , ..., E n _ x ) (C21) 
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n 


P(E H) 
n 


Alternatively# we have 


OCH^, 


E ) = A(E , E (H)O(H) 

n 1 ft 


(C 1 9 ) 


with 


a(e l , 


e |h) = n A(E |H) 

n i-1 


ft P(E 1 |H) 
' "l p ( E ip) 


(C 2 3 ) 


(C 2 4 ) 


vi) Combining Independent A Posteriori 

Probabilities 

Suppose that we had some good method 
for estimating the probability of hypothesis H given just the single 


piece of evidence E. Let 








































Jote that the a posteriori odds increase whenever the odds Dased on 


En alone exceed the a priori odds. It follows that the a posteriori 
probability increases Whenever P(H\En) > P(H). If w e express the 
odds in terms of probabilities, we obtain 


p(h|e x . 




E ) = - 

n 

1 + 


n 

i=l 


1 _ 

Q(H|E 1 )P (H)- 

P(h|e~)Q(H) 


mi 

P (H) 


1 

Q(H|E n )Q(H|E 1 , E n _ 1 )P(H) 

P(H|E n )P(H|E 1 , E nl )Q(H) 


(C 3 J ) 


Note that for the special case n = l 
we merely obtain the identity P(H\Ki) = P(H\fd. The case n = 2 gives 


p(h|e 1 ,e 2 ) = 


_ 1 _ 

q(h|e 2 )Q(h|e i )p(h) 

1 + P(H|E 2 )P(H|E 1 )Q(H) 


(C 3 ? 1 


This can actually be Interpreted as the general case if we think of 
El as the collection of all of the old evidence and E2 as the new 
piece of evidence. The graph in Figure 44 shows how P(H\tl,E2) 
changes with P(H\£2) for the special case P(H) = 1 / 2 . 
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FIGURE 44 THE EFFECT OF NEW INDEPENDENT 

EVIDENCE ON THE A POSTERIORI PROB¬ 
ABILITY OF H 


vil) 


Kelation to Shortliffe's Combining 
tomuja 


Additional variants can be obtained 
bY introducing a quantity C that is analogous to-though not exactly 
the same as—Shortlifte's certainty factor CF. r n general., tor any 
probability p we define C as 


C = 2P - 1 


(C 3 i) 


Thus, as P varies from o to 1, C varies from -1 to 1. After a little 
algebraic manipulation, we can express Eq. cC32) as 
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(. C 3 4 ) 


c<Hl v V =rr 


+ C 2 - C 0 (l + Cl c 2 ) 

°1 C 2 ' C 0 (C l + V 


where 


C = 2P (H |E i > - 1 


(C35) 


C 2 = 2P(H1E 2 ) - 1 


(C3g) 


and 


C Q = 2P(H) - 1 


(C37) 


It is instructive to consider some 
special cases. If all of the C's are strictly between -1 and 1» then 
it is easy to show that 


c (h|e^E 2 ) -* 1 as — 1 

or as C 2 -* 1 

or as C -• -1 






















an>i that 


C(H lE^ E 2 ) -* -1 as C - -1 


or as C - -1 
2 


or as C -* 1 
0 


The limiting behavior tor Cl and C2 seems quite proper* but the 


behavior tor CO at tirst seems paradoxical. if we are virtually 


certain a priori that h is false (CO 


-l)* why should we be 


convinced that H is true a Posteriori? The explanation is that if 


PCH) is approaching zero but PM\E1) or P(H\E2) stay bounded away 


from zero, then the evidence in favor of H is extremely convincing. 


For the special case PM) = 1/2 we 


obtain 


C(H|Ei, E 2 ) = ^ 


c. + c. 


CC3ft) 


or* alternatively, 
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c(h|e 1 ,e 2 ) = c 1 + C 2 (1 - c x ) 

+ [l - c(h|e i; e 2 )] CjC. 


C 1 + c 2 (1 - r i ) 


if C(H\fcl,E2) is near unity. Tnis latter expression is reminiscent 
of Shortllfte's formulas tor updatinq measures of belief and 
disbelief for sequentially acquired evidence. Tnus, to first order, 
we can think of Shortlitfe's formulas as an approximation to Fq. 
(C38), which in turn is for the special case of independent evidence 
with P(H) - 1/2. 


However, the approximation is not 
good if Cl and C2 have opposite signs, corresponding to conflicting 
evidence. For example, suppose that Cl <0 and C2 > 0. In this 
circumstance, Shortlitfe would say that El favors disbelief in ri and 
E2 favors belief in H, and would write 

MB(H, E 1 ) = 0 


MD(H, E x ) = -C 1 


mb(h, e 2 ) « c 2 


mb(h, e 2 ) = 0 


MB(H, \ & E 2 ) - MB(H, E ) = C r 


MD(H, E. 6c E.) - MD (H, E,) - -C 

















provided that neither C2 nor -Cl is unity. The certainty factor is 
obtained as the difference of these measures of belief and disbelief: 

CF(H, E & E ) = C 1 + C 2 

Note that this leads to a curious 
discontinuity in the benavior of the certainty factor. If Cl = -1/2# 

then the largest CF can become as C2 accroaches unity is 1/2: 
however, in the special case C2 = 1, ShorclMfe sets MD(h, LI f, E2) 
to zero and obtains CF = i. mis av*w d rd limiting behavior is 
avoided by tlq. C C 3 8) (and by the other, more general formulas we have 
derived), in which C(H\Ei,£2) approaches unity continuously as C2 
approaches unity. This win be true tor any value of Cl except Cl = 
-1. In general, one must expect anomalous behavior when Doth 
conflicting conclusions are certain, but there seems to be no reason 
to accept discontinuities in noncontradictory situations. 


Vision 

1• Introduction 

The vision modules associated with the pointing 
system described in Section 11.C were designed for limited and well 
defined purposes. Our lonq-term objective is considerably more 
ambitious: the construction of a visual information gathering 

facility that may be called on by other subsystems to answer 
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questions from the apprentice* monitor his performance* and verity 
and update the knowledge base. This facility will consist of a large 
number of general arid special purpose vision modules coordinated ny e 
problem solving executive. This executive will use knowledae of the 
task context* of the perceptual domain, and of the available 
perceptual capabilities* to plan effective strategies for fulfilling 
information requests. This section descrioes two major research 
efforts, one on perceptual strategies, the other on scene 
understanding* directed towards this long range objective. 

The techniques required for fulfilling information 
requests may range from a simple table lookup to a major analysis of 
the entire scene. Certain common requests may be handled directly by 
Passing them to a Packaged module such as a tool recognizer. However* 
many tasks will undoubtedly require planning at a much lower level. 
Consider a question such as "is the flywheel on backwards." It so 
happens that the flywheel on our compressor is Painted only on its 
front surface. Knowing this tact* the particular question posed 
above can be answered relatively easily by testinq color at the 
predicted image location of the flywheel, (The prediction could be 
based on the same compressor model used for pointing.) Obviously it 
is impossible to provide canned strategies of this type for all 
questions the apprentice might reasonably ask. 

The issues entailed in planning perceptual strategies 
are fundamental ones that arise in every complex perceptual domain 










regardless ol the particular modules Involved. First the available 
ootions tor accomrlishina the task must p e assembled in a Planning 
draph. Second, tne options must be orqanlzed into a plan of action 
based on relative cost and likelihood ot achieving desired results, 
lhirdr the Plan must be monitored during execution to detect 
departures from initial planning assumptions and, if necessary* be 
revised. For several years, we have been Investiaating these issues 
in the context of strateoles for finding objects In loom scenes t37, 


54) . 

There are similarities between strateqies for object 
finding and those tor question answering; the same techniques that 
select features for distinguishing a chair from other room objects 
can be used for distinguishing the front and back of a pulley. in 
the next subsection w e describe an implemented system that can Plan 
cost-effective information gathering strategies in a variety of 
domains . 

Tne second area ot research concerns scene 
understanding. Machinery is one mf the most difficult scene analysis 
domains. Components on a given mechanical assembly often cannot be 
distinguished by any local characteristics except their detailed 
shape. Moreover, visual character1stics may vary widely over a 
generic class ot components performing the same function (e.a., 
carburetors). Part identification is thus likely to result in 
ambiguities when the parts are viewed in Isolation. Fortunately, a 
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rich set of constraints governs relationship of parts to each other 
in a complete assembly. These constraints may arise from 
consideration of the function or operation of the parts, or from 
methods of fabrication or assembly. For example, carburetors are 
connected to the intake manifold, a fuel line* and a throttle 
linkaqe. The absence of neighboring carts that could reasonably 
assume these interpretations makes it unlikely tnat a giver part is a 
carburetor. Our goal is to use such general knowledge aboJt a class 
of equipment, (e.g., automobile engines) to resolve interpretation 
ambiguities on individual parts, eliminating the need for detailed 
structural models of specific units. 

A framework for performing this type of reasoning 
about scenes was described in last year's final report. Since then, 
the design has been refined, implemented, and tested in the room 
scene domain. Hoorn scenes were used for experimental convenience in 
testing the basic reasoning system because representations for both 
local features and global relations were already available. 
Comparable representations are currently being developed for the 
mechanical equipment domain with local interpretations based 
primarily on 3 -d shape inferred from range data. This work is 
reviewed in a subsequent subsection. 

2. Scene Analysis Plan Generation and txecution 

We have designed and implemented a planning system 
that makes use of Information about anticipated costs and 














5 reliabilities to choose the best course of action. Although the 
Planner was designed ana implemented as a core subsystem of a vision 
system that locates onlects In office scenes 137, bb-b?). ae feel 

that the applicability of the process reaches further than tne 
boundaries of scene analysis, therefore, »nlle tne examples win 

mostly be chosen from problems m that domain, tne Presentation will 
be of a rrore gereral nature. 

-or artificial intelligence pronlem solvers, the real 
problem has traditionally been to generate any Plan for performing 
the desired task; since the choices tor solving the problem were 
generally limited, the choice of wnlch subooal to work on and 
subsequent execution has been the easy part. The plannel to oe 
described here generates a (relatively) complete plan for the 
analysis of a scene, hut then, more Importantly, organizes tne Plan 
so as to know what to work on and In what order. The problem at any 
Point in execution Is to know what to do next. An example of a slmp le 
Plan for finding a chair Is shown In Figure 4b. we will into 
detail about the generation of this plan below. 


The planner first elaborates tne goal Into the 
subgoals required for its solution. Tnese subgoals are elaborated 
in turn until no more remain. The result Is an AhU.-OK planning graph 
with tip nodes that are executable subgoals. Next, the scorer 
organizes the graph so that tne executor can find the best terminal 
node to beam evaluating. rne plan executor executes the best 






■ .> 


m.V 


vvv-v-.y 


iaifc is* > tm ^ ^ ■« - 


■ . • v - u. >• a - 

-Ve’uV.V\-V".".V 






kia 


*-« —*1 -*w lisiwl.F w ■ ■ - a - 


. • . V* 







































sdbqoai and then propagates the success or failure of the goal 
through the original plan, in order to select subsequent subaoals. 
txecutior continues until the top goal either succeeds or fails, at 
which time the process of planning and execution is complete, 

a. Definitions 

Before describing how the planner works, we 
will present some definitions and descriptions of strategies, plans, 
and elements of plans, the most important of wnich is tne goal node. 

This node contains information about the satisfaction of the goal, 
and also contains (implicitly} the complete structure of the plan. 

Goal - a representation of some state to be achieved. A goal is 

either directly realizable (i.e., executable), or is an intermediate 
goal, which is satisfied by satisfying its subqoais, An 
executable goal Is a terminal goal. 

The format of a goal is a list, • ,ie first element of which is usually 
the activity represented by the goal (e.g., FIND, VALIDATE). The 
second element is usually the object for which the goal is being 
executed. Succeeding elements (if any) usually contain 
information Pertinent to the goal. For example, 


















(FILTER-ViNDOW DOOR 


(LAMBDA (X) (LIMITP X (FUNCTION HEIGHT) 

2.5 5.0)) 

(LIMITP X (FUNCTION SAT) 

.6 .75))) 

((HEIGHT (6 . 8)) 

(SAT (7 . 12)))) 

is a typical qoal. The activity is F1LrtR°«lNDOW, a filtering 
proaram that passes a LISP predicate over a sampled window of 
the image; the object is DOOR; the next element is the LISP 
predicate to be used in filtering# and the last element is the 
Internal representation of that predicate. The last two elements 
are particular to FILiER-^lNDUW, 

Subgoal List - a list of subgoals preceded by an operator from the set: 

AnD# #AnD, oh# or t. The operator Indicates how solution of the 
subgoals relates to solution of the goal, AND and #AnD imply that 
all the subqoals must be achieved to satisfy the goal; OR implies 
that only one of the subqoals must be satisfied. A * AND requires 
that the subgoals be achieved in sequence, t is a dummy operator 
that is used when there is only one subgoal; it means that 
satisfying the subqoal is equivalent to satisfying the goal. 

This operator is used primarily to minimize the complexity of 
the subqoal lists. Some example subgoal lists are; 


























Node 


Plan 


(AND (FIND StAT)(FIND BACK)), 

(#AND (ACQUIRE DOOR)(GROW DOOR)), and 

(OR (ACUU1RE-UIRECT PICTURt) (AC QlJlRt-InJ DIRECT PICTURE)). 

The first example is the suogoal list for (FIND CHAIR); 
the second for (Bound DouR); the third for (ACQUIRE PICTURE), 

(or goal node) is a list consisting of tne goal, the subgoal 
list for the qoal, a set of parameter lists for scoring 
purposes, and the parents of the qoal. For example, 

((ACQUIRE DOOR) (OR (AC QUIRE-DIRECT DOOR) 

(ACQU1RE-1NDIRECT DOOR)) 

(l«3) (.95) ( • P55 ) (50000.) (58500.)) 

((HOUND DOOR))) 

could be the qoal node for the goal (ACQUIRE DOOR). 

the set of goal nodes generated for satisfylna the 
Initial goal, in Its most general form, the plan Is an implicit 
AND-UR graph, since the subgoals point to one another through 
their own subgor.l lists. The graph structure appears when a 
goal has some other for a subgoal, which in turn has 
the first as one of its subgoals (althouon not necessarily 
as a direct descendant),. For example, as shown In Figure 45, 
to find the seat of a chair it might be Possible to first find 
the baclc and use that to localize the seat. However, to find 
the back, it mignt also be possible to first find the seat and 

then localize the back. This sort of reasoning lads to plans 
with loops. 
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Module - a module Is a program or a statement describina how to satisfy 

a goal. In the case of executable goals, the module Is an execution 
module and is the actual program to be executed. For nonterminal 
goals, the module Is a planning module that describes how the 
goal is to be expanded Into subzoals. A module may have various 
functions associated with it. Some common ones are the CliSTFN 
(which computes the cost of execution of the module), and 
the PTFN (which computes the probability of success). In addition, 
the module may also have associated functions for deciding if a 
previously generated goal is equivalent, or related. 

Cost - in the context of planning, the cost is the anticipated time 

that would be spent trying to satisfy a goal. In the case of 
bounded processes, such as filtering, the cost can be estimated 
rather closely. For less bounded processes, such as reqlon 
growth, the cost is approximated from Past experience and 
whatever relevant Information may be available (tor example, 
the expected size of the region to be grown). 

Confidence - In the context of planning is the probability that the 
execution of the goal will have a good outcome. l’hls is 
composed of two Parts, the probability that the outcome 
will be good, given tnat the node succeeds, and the 
probability of success of the goal. Again, in cases where 
the process is well modeled, the estimates are relatively 
good, and where there Is no good model, the estimates 
are based largely on past e>:p*»rlence. Confidence 
is always between 1,0 and 0.0. 











Score - the score of a node is the value of a function that computes 
a cost tlqure normalized by confidence. For example, cost 
divided by confidence is the measure that w e use here; 
however, other functions are being experimented with, 

b• Plan Elaborator 

To create a plan, the plan elaborator (Pt) 
expands the initial qoal (the "top goal", which is typically to find 

some object, e.g., FIfJD( l'ABLE)) into the subgoals required tor its 
solution. The subgoals are recursively elaborated in exactly the 
same way. if the Pt cannot generate suoooa*s, it checks to see 
whether the goal is directly executable, and if so computes 
parameters (cost and confidence) for later use in scoring. If it is 

not directly executable and has no subgoals, the elaborator marks it 
for later deletion. 

As new objects are noticed by tne PE, their 
names are added to an "instance list". This list is used as a global 
"note-pad" to keep track of w n at is learned about the objects during 
execution of the plan. The initial entries into the instance list are 
taken from a globally maintained list 0 f prototypes; for example, 
-hen the chair seat is noticed for the first time in the course of 
Planning, the name is looked up on the list of prototypes, this entry 
is then added to the instance list. The prototype contains general 
Information such as relations between tne object and other objects, 
or previous plans that were created. This information is available to 

















the planner durinq elaboration, and is also used by modules durinq 
execution as a stor?.oe medium containing up-to-date Information about 
previously executed modules. For example# the chair seat prototype 
could contain the information that the chair seat is below the chair 
back. During execution of a plan to find the chair seat# the 
acquistion module (discussed below) would save the acquired chair 
seat samples on the instance list, whlcn would then be available to 
the validation module for further processing. 

To aenerate the suoqoals of a goal# the 
elaborator checks first to see if any special modules exist for 
satisfying the goal for that particular object. This allows the user 
to provide advice to the Planner about certain goals, lor instance# 
the planner could be instructed to always look for the telephone on 
the tabletop. If no special routines are available, the planner 
looks for routines associated with the activity and that are 
generally useful for all objects. 

For example, to elaborate the goal (ACQUIRE 
TABLE), the planner might be instructed to use a horizontal plane 
finder, since the tabletop is a horizontal Plane, Alternatively, if 
there was no information associated with TABLE, the elaborator would 
look into general purpose routines Pertinent to ACQUIRE, Hqure 45 
provides another example. There is a module for KIND that says to 
find an object, find its parts. Therefore, to find the chair a Plan 
is generated for finding its major parts, the seat and the back. 

















In aener a l, several planning modules may be 
associated with a given activity. in order to select the expansion 
modules appropriate to a Darticular goal, tne Planner evaluates a 
predicate function associated with each. if tne predicate returns a 
true value, the planner adds the module to its list of applicable 
modules. Special cases of the predicate function can cause the 
module always to be selected (to provide defaults) or can cause the 
module to be selected by goals specified explicitly. This predicate 
allows us to have several modules, each an expert in a relatively 
narrow domain, and eacn selectable on tne basis of that domain. For 
example, associated with the activity, ACQUIKL, could be a program 
for locating horizontal planes. This program would then have a 
predicate that checked that the object was horizontal. 

Before a newly generated goal is added to the 
list of goals to be elaborated, a check is made to see if it has 
already been generated, or it any equivalent supgoals have been 
generated (the subgoal must have a special function to allow the 
Planner to decide this). In either event, it is not processed 
further, and the subgoal list of the goal being expanded is made to 
point to the previous goal. The planner win also check for related 
subgoals (that can succeed or fall based on other goals that are not 
directly related, j.e., not parents or children). For example, the 
Planner may generate a strategy that includes multiple filter 
subgoals where one filter predicate subsumes another. It the less 
restrictive predicate fails to turn up the desired object, it is 












guaranteed that the more restrictive one also Kill not. iherefore# 
if the less restrictive filter is selected and executed# its outcome 
co Ld imply the outcome of the related, restrictive filter. Tne 
Planner maintains an explicit list of related subqoais# tor tnis type 
of situation. 

c. Scorer 

After a plan has been elabbrated, the next 
step is to decide which subgoal to attempt# or# in general# to decioe 
what to do next. Obviously# the choice depends on the local expense 
of executinq the goal, the more global expense due to the fact that 
sor.ie goals may be easier after other goals have been executed# and 
also the resuitina likelihood of correct results after the qoal has 
been executed. 

we compute a score for a node which allows us 
to ma<e a selection from proposed alternatives for satisfying the 
goal. This score function will be cost normalized by confidence# with 
the result that high confidence# low cost goals win have a smaller 
score than low confidence# high cost goals# and medium confidence# 
medium cost goals will fall somewhere in between. 

Before discussing the techniques of Scorinq a 
plan# we digress briefly for a preview of plan execution. The plan 
executor receives a completely scored plan. It starts at the top node 
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at the rlan and proceeds jo*n*ur.i through tne Plan, *1*4/5 seiertinq 
the best oranch (on tne rasis ot score) until it reaches a terminal 
hOde. J’hlS executor le coal is tnen evaluated. It tne siipooal fails, 
that failure is reqaroeii as arseijte, ard tne result is propeqated 
oacfcwards through tne uhvi, no*pvei» it tr.e none suoceeis* then t^-p 
contioence ot tne none is prooaaatei rac-c. That is, «,ii nodes 
oepenoer.t on the successtul n 0 -)e h.,vp their inticioateo confidence 
altered to reflect the success ot tne none. -.her tr e rropauation 
ceases, if the tor cinal has not ne>»n s itisf i-? i tne undated Plan is 
resected and execution croceeis as above. 


ioe output of th<» pi an e labor a tor is a 
Planning graph. Inis representation ot fne strategy* although 
usually ronreentrant, ma/, in general, nave locos. while the 
Proolem of backing costs mo confidence up tnrouun a tree structure 
is quite simple, the sane problem in a graph *itn locos is not quite 
so straight forwarn. i-e -v til first describe the technique tor a tree, 
and tnen Show how it applies to a oraon. 



ine scorinj process isps i relaxation 
technique. All noals in tr» plan are put on a Itsi to pe examinee. 
For each one In turn, tne score is coT.buteg (it cossiniej. If tne 
score of a node changes fret, the last valje, its rarents are nut on 
tne list to have a new score calculated. r.ie process continues 
until the scores stabilize. If a none nas u score, it 'i-eans that 
some way (presumably) exists lc satisfy tnat ir. q. 
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Since the scores tor terminal (executanle) 
nodes do not change/ the only thing *e need consider tor scorina 
purposes are the intermediate nodes. we oeqin the discussion by 
considering the way scores are bac<ed-up from a subnode to its 
immediate predecessor. 

There are four possible types of intermediate 
nodes, *ANL>, AND, UK, and t. The t node merely passes its score 
upwards to its parents/ and therefore need not be considered further. 
Since the *ANL) and AdD nodes require all the subgoals to be 
satisfied, a score for this type of node can only be computed if 
scores for all the immediate subgoals have been computed. However, an 
UK node merely requires that one of the subqoals be satisfied. This 
means that even it only one ot the suogoals can be scored, the UR 
node can be scored. Effectively/ any unscored node is considered to 
nave a score of infinity. The score of an AM) or *AND node witn an 
unscored subqoal is also infinite, unscored subgoals of UK nodes can 
be ignored, and tne score computed on tne basis of the ones already 
scored. In practice/ what occurs is that an initial score is 
determined for some supaoals, which allows other subgoals to receive 
scores. These subgoals In turn propagate their scores back/ and the 
process continues to completion. At the end of the process, any 
nodes that have not received scores are deemed impossible to satisfy/ 
and are deleted from tne plan. In operation/ tne actual parameters 
tnat are backed up are cost and confidence, and the score of the node 
is computed from them directly. 























we will use the symbol S to stand for 
score, C for confidence, and k tor cost, we wm also use 
T for tne successful execution of a node, T k for the sequence 
of successful executions of nodes 1 tnrouqh x, T for 
unsuccessful execution of a node, and 


for the sequence of unsuccessful executions of nodes 1 through k. 
Tnus, s. is the score of the itn node, iq the anticipated 
cost, and c t the confidence. T. is the representation of 
successful completion of node i. 

Tne confidence is the a priori probability of 
a "good outcome" from the none. Ubviousiy, the notion of a good 
outcome is directly related to the task the node is supposed to 
satisfy. For example, a qood outcome from a filtering module would be 
that it discovered a few points on the target oblect. ihe a priori 
confidence requires that the node succeed for a good outcome to 
result, and is therefore the Joint probability of a good outcome and 
success of the node. w e make the standard Bayes' expansion to 
represent confidence as the product of two factors: the probablity of 
a good outcome from the node, given the success of the node, and the 
Probability of success of the node. If w e let y ± represent a good 
outcome from the 1th node, then we have 

c i = P(Y i |T i )P(T.) . 
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We will also note that the a posteriori probability of a good outcome 
from a node is zero if it failed, and 

PCyJt.) 

if it succeeded. 

As mentioned above, the score of a node will 
be the cost normalized by the confidence, that is 

S = K/C 

for the score function. In the course of searching for the best path 
through the qraPh, we win attempt to minimize this function at each 
node. In order to compute the score of intermediate nodes, we will 
back-up costs and confidences from subnodes to their superiors, we 
will discuss techniques for that now. 

1) scoring an UP Node 

we begin the discussion with the OP 
node in Figure 4b. We assume that the suonodes are ordered by 
increasing score. The backed-up parameters will be dependent on the 
order* and it can be shown tnat this particular ordering will result 
in a minimum score tor the node. 































For an UR node to nave a good 
outcome, one of the sutnodes must have a good outcome. Therefore, we 
can write the confidence as tne sum of the orobaoiiitles of a good 
outcome from each sequential node, lhat Is, 

= P (V. ,T ) + P (Y_. t . t 7) + ... + pfv . T . T i ~ 1 \ 

















Expanding this formula gives 



n _ __ 

£ p (V i |T., T i_1 )PfT 1 |T 1 " 1 )P(T 1 " 1 ) 


(o n 


We have In 


P(Y., T., T 1 ' 1 ) 

a context sensitive measure of confidence. That Is# It Is 

confidence measure that taxes the past history of the parent noc 

into account. As this will, In general# De difficult to compute, w 

will maXe the weaXenina assumption of independence between T. an 
. , i 

T i-1 

1 , and therefore 


T 


i-1 


we also assume that 


is dependent only on T i , not on 


with this assumption, we can rewrite the formula as. 


M 


"OR 


- E p <v 

i=l 


T )P (I 1 " 1 ) 


(04) 
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( 05 ) 


C 


OR 


=- y c. p(t 1-1 ) 
i-i 1 


And finally we have 


OR 


n i-1 

= E C - n & - P(T.) ] 

i=l J-l J 


( 06 ) 


The expected cost of the node Is tn? 
sum of the expected costs of each of the Individual subnodes, wnich 
Is just the cost of the node times the probability of executing It, 
The probability of executing a node Is the probability that all 
preceding nodes fall. Therefore, we can write the expected cost of 
the node as 


n 



i-1 


K. P(T i_1 ) 


( 07 ) 


With the same assumptions of Independence as above, we can write the 
cost formula as 


n i-i 

K qr E k - n [1 - P( T .)] 


( 08 ) 













The proDdbility ot success of the l)R 


node Is just 


1 - P(T n ) , 


tnat is, the probabliitv that they do not all fall. This Is rewritten 


p (i„) = i-n 

0R i=i 


n - p(t.)] 


(uy j 


The score is computed from the 
backed-up cost and confidence, it is apparent, however, that a very 
expensive subnode with low reliability would pull the score of the 
whole node down, even though it might be likely that it would never 
oe executed. Since we would like the score of a node to reflect the 
best that we can expect from the node, we will define tne score to oe 
the minimum score ot the set of subsequences of the sequence of 
subgoals. That is, we can compute the score for the whole sequence, 
then tor the subsequence without the last goal, then without the last 
two subqoals» and so on. We then take the minimum of these ..cores to 
oe the scure of the node. But this definition implies that the score 
of an OR node will always be the score of the first subgoal in the 
sequence. Therefore, the first subgoal in the sequence win represent 
the node. As a result, the planning parameter^ tor the node become: 






















It Is important to note that this result is strictly due to the 
choice of score function. Ihe more general results will be required 
in later work with different score functions. 

Ill Scoring an and Node 

ke assume that the subnodes of an AND 

node have been ordered by 

yu - c^, 

as this can be shown to result in a minimum score for the node. A 
*AN0 node is already ordered, so the formulas derived for an AND node 

"HI aPPly eqUally to a 'AND node. The And node is shown in Figure 
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FIGURE 47 BACKED-UP PLANNING PARAMETERS FOR AND AND *AND NODES 


The confidence in this case Is 
probability that all the subgoals will have a good outcome, or 


C ANn = P < Y "' ^ = P(Y n |T n )P(T n ) 




with our assumptions of inde endence, we have 


P(Y n |T n ) = H P(Y.|T.) 
i=l 


P (T ) = [1 P(T.) 
























Therefore, 


AND 


n p(y. |tj 
1=1 1 1 


II P(T.) 
Li-1 1 


(L» 1 2) 


or 


°AND = 11 P(Y ^ P(T ^ 
i=l 


l l 


(D 1 J) 


And, therefore. 


AND 


n 

= n c 


i=l 


(D1 4 ) 


If a suonode of an AND node falls, we 
terminate execution. Therefore, the expected cost of a single 
suonode In the sequence Is the cost of the node times the prooauinty 
of executing it. This proDdDlilty is the probability of all preceding 
nodes succeeding, Tne cost, therefore, is 


n 


%> ■ £ 
i-1 


( D15 ) 
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The probability of success of the 


node is 


P( W = n P(T.) 
i=l 


( D1 7 ) 


The new score is computed from these backed-up values. 


111) Scoring a Plan 


with these techniques for computina 
the planning parameters of individual nodes# organizing the planning 
tree becomes a straightforward task, Tne Parameters of the terminal 
nodes are backed-up to their predecessors. These intermediate nodes, 
in turn# pass their parameters on to their parents. Finally# the top 
node receives a score and the process is complete, ' 


However, the process of scoring a 
plan with loops (such as the one in Figure 45) is more complicated. 


[»!•] 







































me program tnac scores a uran 
assuires that the Initial score of a nonterminal node Is infinite. An 


AND (or *AND or t ) node can receive a noninfinite score only it all 
of its suogoals have noninfinite scores. An UR node can receive a 
nonlrflnlte score it any of its subgoals has a nonintinite score. 


Therefore, in the graph, nodes 7 and 8 can receive initial scores of 


V c i 


K./C 
3 3 


respectively. Using the parameters from node 7, node b can be 
scored. It the score for node 6 is less than 


k 3 / c 3 , 


tnen node 8 receives a new score, in any case, node 5 can also 
receive a score since node 8 now has one. Again, if the score of 
node S is less than that for node 1, tnen node 7 receives a new 
score, node b receives a new score, and node H receives a new score. 
It is important to notice that it the score of node 5 is less than 
that tor node 1, and this is reflected oack to node 6, it is 
impossible for the new score of node 6 to be less tnan that for node 
3. If It were less, it would mean that costs around the loop 
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decreased and/or confidence Increased, whicn is not Possible. 
Therefore, either the final score of node 5 will be greater than that 
of node 1 or the tl lal score of node b will be greater than that of 
node 3, we also see that tnere will be a maximum of two iterations 
around the loop before things settle down. 

To oe more precise, we win tertorm 
tne computations with symbolic quantities for .he parameters of 

terminal nodes 1, 2, 3, and 4. the initia score for node 7, 

is 


From this, we can compute 


S 6 " 


K 1 + P . K 4 

C 1 C 4 


( 1 / 18 ) 


and, therefore, 


S = min 

O 


K_/C , 
3 3' 


K 1 + P 1 K 4 
C 1 C 4 


(1/ J ^ ) 
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for K and A greater than u and C and B oetween 0 and 1 


S ? = min 


W 


K 3 + P 3 K 2 
C 3 C 2 


since if 


*1 + P 3 K 2 
C 3 C 2 



S 8 ' V C 3 


s 

s 


K + P„K 
3 3 2 

C C 
3 2 


The final result is 


min K /C 


K i + P A 










































and 


K 1 + P 1 K 4 


c c 

14 


d. 


Execution 


After plan scoring is complete* tne execution 
program selects tne best executable subgoal, and evaluates it. Tne 
success or failure is propagated upwards to all parents# and anv 
related subgoals are dealt with also. It tne strategy has not 
succeeded or failed at tnls point, the planner calls the scorer to 
rescore the net and then continues with tne next goal. 


To do tnls, the executor starts at the top 
goal of the net, and proceeds downwards through the subgoals. 
Wherever there Is an option. It chooses the subgoal with the lowest 
score, bhen it finds a node where the pest supgoal is executable. It 
passes that subgoal to an evaluation function. This function 
executes the subgoal and propagates the success or failure Backwards. 
It keeps track of subgoals that were executed, along with their 
outcomes on a special list. 
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n c./ n p(t.) 

• i 1 • i 1 

i=i 1=1 


The remaining cost after success of the kth node is 


n i-1 

E K - n p ( T -) 

i=k+l 1 j=k+l J 


Since the confidence is i n cr sJng, and the cost decreasing, the 
score of the node decreases. This means that the subgoals of the 
node win be executed in order as long as they are successful. The 
final confidence of a successful A vD node is 


n n 

n c/ri p(t) 

i=l 1 1=1 1 


If a subqoal of an UP node succeeds, then tne OH node Itself succeeds 
with a posteriori confidence, 


C/P(T). 

This is propaqated back alonu with the cost (which is ze.j) and 
P(T) (which is one) to the Parents of the OR node. 
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After propagating tne outcore of a goal, the 
executor checks the list of related goals to see if any other action 
needs to te taken. Since in many cases tto outcome of cne qoal can 
determine tne outcome ot another, tnis is tne point at which action 
Is taken. 

If the outcome of a subgonl does not 
propaqate to the top goal, the net has a new score computed tor it in 
exactly the same way as it did initially, and the execution continues 
with the next qoal. It, at some point, a graph with a loot) is left, 
and there are no terminal nodes that wm allow tor a solution, then 
no failure will propagate to the top. Instead, tne top goal will not 
receive a score. This is considered a failure of the top goal. 

e. Conclusions 

we have described an artificial intelligence 
Planning and execution program that is able to make cost effective 
decisions about what to do next in analysing scenes. w» have 
described in some detail the effects of a particular score function 
(i.e., K/c , This is probably not tne best rossible score 
function. Another possibility that wm p e explored is to treat an 
QR node as if there were a discrete cost-continence function 
associated with it. That is, considering each subqoal, there is a 
certain reliability to bt? expected for a certain cost. we can 
consider these cost versus reliability values as points on a curve, 














and the complete set provides a cost-confioence function. This 
function can then be propaaated upward instead of making a decision 
aoout what the score shoula he at tne local node level. Idis *a/, 
the parameters can he passed up to the executor which can tnen n e cio e 
(uslni a variety of criteria) just what should be none. With this 
kind of information available, the executor couio use budgetary 
considerations (how "ucr, resource is available to expend tor the jot) 
or considerations about required reliability. 

• Ksis: a oyster tor Reasoning aoout Scenes* 

a. Introduction 

Tois section describes ^syS, an operational 
system that interprets scenes. Scene interpretation is formulated as 
a constraint optimization proolem. Given a set of regions from a 
Partitioned s ce ne, a s e t of possible Interpretations for each region 
derlveu from local surface attributes (e.q., color, orientation), and 
a set of constraints on spatial relationships of interpretations, 
<SYS deduces the most probaole global interoretation for each region. 
Tne systerr nas been used to interpret manually partitioned room, 
scenes usinu a variety of semantic constraints. 


♦The work reported in tnis section was partially 
suppcrted by DNR under Contract N 0 . Noo14-71-C-0294 [b8J 









.Scene interpretation has open previously 
approached both as an exercise in deductive reasonino and as a 
problem In heuristic optimization. As a reason! ia exercise* the 
objective was to deduce unique local interpretations for each reqion 
that were consistent with symbolic glooal constraints 159-61], As an 
optimization problem, the oojective was to find the set of local 
interpretations with the highest joint liKelihood* where each 
interpretation llkelinood was a function ooth of the local reaion 
attributes and the ll.celinoods of semantically related 
interpretations assigned to other regions [62-61J. 


The most successful deductive scene 
interpretation system, was created by /*altz 161) for analyzing ideal 
line drawings of scenes containing toy nlocks. Lines were labeled at 


each vertex with a set of possiole interpretations, such as concave, 
convex, shadow, and cracK. Inese initial interpretations were based 
on a catalog ilstlnq the various ways legal vertices of trihefral 
solids could appear in a line drawing. Some line interpretations 
could be immediately rejected as impossible if they w e re not assigned 
at both ends of the line, fclimlnatina a line interpretation would 
eliminate one cr more possible interpretations for eacn of its 
vertices, wnlch in turn, could eliminate additional interpretations 
of other lines joinlnq at those vertices. This elimination process 
would often propagate until each line was left with a unique 
Interpretation. 
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ine deductive approacn to scene analysis is 
limited to symbolic problem domains such as line drawing analysis, m 
which all Interpretations that violate constraints are dismissed 
absoluttly and all those tnat do not are considered equally likely, 
lost real perception problems, ty contrast, entail noisy measurements 
and probabilistic constraints. Thus, it is not meaningful to 
speak of correct or incorrect interpretations, out only of the most 
probable Interpretations, given current evlaence. A number of 
investiqators have therefore chosen to view scene interoretation as a 
heuristic optimization problem. 

In seer,? Interpretation, the likelihood of 
each region interpretation theoretically depends, at least 
indirectly, on the likelihood of every other region interpretation. 
Because of this interaction, the search space explodes exponentially 
with both the number of regions and the universe of possiole 
interpretations. Numerous simplifying assumptions have been proposed 
to make the search tractable. Both puda [ 59 ] and Guzman 160) 
performed a limited tree search using local binary valued constraints 
on legal adjacency: interpretations were assigned to regions in order 
of maximum iocal likelihood# suoject to semantic consistency with 
interpretations previously selected for adjacent regions. Elimination 
all possible interpretations for an unassiqned region forced 
reconsideration of an earlier assignment. Yakimovsky developed tw 0 
algorithms which, while still based L-n tree search, utilized wider 
classes of constraints C63J. Algorithm a allowed real valued 













boundary constraints which could alter the local livelihood of an 
interpretation based on the attributes and interDretation livelihoods 
of adjacent reqions. These constraints were used tollowin an 
instantiation to update the interpretation livelihoods of 
uninstanfiated regions immediately adjacent to the newly instantiated 
one. A branch and bound condition on the product of all instantiated 
interpretations was used to terminate search alona umbromising lines. 
Algorithm B allowed real valued constraints between arbitrary pairs 
of region interpretations. Tnese constraints were again used only to 
update interpretation livelihoods of uninstantiated regions, that 
were directly related to a newly instantiated interpretation. The 
use of real valued constraints provided Yavimovsvy with a more 
sensitive measure of global livelihood for assigning Interpretations 
to regions. However, since both his algorithms limited constraint 
interaction (by freezing the livelihoods of instantiated 
Interpretations) in the Interests of search efficiency, the resulting 
likelihoods represent only an approximate simultaneous solution of 
the constraints. 


Besides obtaining only approximate solutions, 
all of the above methods share another inherent fault; hard won 
information is frequently thrown away when the search bacvtracks. in 
one of Duda's examples, a particular region is deduced not to be an 
electrical outlet on at least five separate branches of the 
interpretation tree. In each context, outlet was excluded as a 
Possible interpretation because the region in question w a s adjacent 
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to a region that had already been postulated to oe the floor, it was 
precisely this type of redundant reasoning tnat waltz sought to 
avoid, by eliminating inconsistent interpretation before embarking on 
search. It occurred to us that a similar deductive scheme in* 11 . he 
used with real valued constraints to depress the likelihood of 
inconsistent interpretations. An interpretation witn low enough 
likelihood would effectively be eliminated because it would never h » 
selected for instantiation. The consequent reduction in wasted search 
effort could then ue redirected toward i more optimal ulobai 
determination of interpretation likelihoods. 


The following section describes a realization 


of these objectives. 


- An Optimization Algorithm Usina 
Constraint Propagation 


This section outlines an algorithm for scene 
Interpretation that combines the best features of deduction and 
optimization 158]. bach region in a partitioned scene is assigned all 
interpretations Permitted by its local attributes (p.g., it may be 
doo- 01 wall on the basis of color, sizp, and surface orientation). A 
likelihood is associated with each interpretation, -based initially on 
the a priori likelihoods 0 f alternative interpretations for the 
region. These local likelihoods are then reevaluated based on the 
cuxrent likelihoods of all semantically related interpretations. This 
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simultaneous reevaluation is accomp 1 ished on a serial computer by a 
relaxation process that culminates with a set of global equilibrium 


likelihoods. This relaxation process Is the analog of Waltz's 
deduction process tor rtii valued constraints and, like that process, 
can be Implemented using a propagation technique, as *111 be 
discussed In Subsection c below. 


lhe resulting equilibrium likelihoods 


represent estimates of the global validity of alternative 
Interpretations In the current context. It all regions are left with 
a single probable interpretation, the analysis terminates 


successfully. 


Un the other hand, if all interpretations of any 


reqions are Judged unlikely, the analysis Is abandoned on grounds 
that the context is inconsistent. The most usual case finds at least 
o:e region still left with several possible interpretations. The 
analysis then reverts to a searcn for the set of region 
interpretations having the highest joint likelihood. 


The search proceeds at each step by restorlna 
the highest scoring context (initially the global context) and 


instantiating the most likely ambiguous interpretation remaining in 
that context. Equilibrium likelihoods are recomputed, based on the 
new instantiation, and used, as in the global context, to decide the 


future course of the search. If all regions now have unique 
Interpretations, the search Is terminated, it any region has no 
possible interpretations, the context is abandoned, otherwise, a 
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qlobal score for tne context Is computed Dy sjmminq the equilibrium 


likelihoods of the currently best interpretation for each renlon. The 


search then continues in the currently best context. 


In Ref. It Is proved that this alaorithm. 


*lth reasonable restrictions on tne nature of the constraints, will 


terminate with the optimal set of interpretations for those 


constraints. The resulting equilibrium likelihoods will, in general. 


be better estimates of actual global likelihoods than tnose obtained 


by the methods described in Subsection a above. Moreover, it Is shown 


that, in finding the optimum, tnis new Search will instantiate no 


more interpretations than any of tne conventional tree searcnino 


methods. The improved estimates of interpretation likelihood provided 


by the global relaxation process imorov e the order ot instantiation 


and context selection at each stage of search, thereby minimizing 


backtracking. The relaxation process can thus r,e considered a new 


type of glopal lookahead embedded in a conventional best first search 


algorithm. All of the alaorlthms discussed in Subsection a are in 


fact special cases of our algorithm utilizinn limited propagation 


and/or constraints 1 5BJ. 


The number of instantiat1ons is, of course. 


only one measure of search efficiency. 


1 he computational effort 


expended in lookahead must also be considered in assessing which of 


two optimization approaches is more efficient overali. 


relaxation procedure is, admittedly, a computationally expensive form 
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ot lookanead, but it can be very cost-ettective in problem domains 
with large numbers ot hypotheses interacting tnrouah many oair*ise 
constraints. This cost-effectiveness stems from thp fact that many 
inconsistencies are deduced once and for all in the alobal context 
rather than havina to be repeatedly rediscovered tor individual 
instantiations as in conventional backtrack searches. 

c. MSYS - An Implementation of M* 

Implementation of the M* optimization 
algorithm described above requires a relaxation mechanism for 
constraint evaluation and a context switching mechanism for searcn. 

We win describe generalized implementations for these two 
components* which are designed to facilitate experimentation with a 
variety of knowledge representations, search strategies, and control 
structures (including specifically all strategies mentioned in 
Subsection a). 

i) Constraint tvaluation--The XDEMUN 
System 

The simultaneous determination ot 
equilibrium likelihoods tor a mutually constrained set ot 
interpretations, inherently a parallel operation, has Deen 

*A detailed Implementation ot MSYS using these components appears 
in Ref. 58, 










efficiently Implemented on a serial computer by simulating a network 
of asynchronous parallel processes interacting through a gloDal data 
base. The data base consists of variables representing constrained 
entities and constraints. Associated with each variable is a 

procedure for computing a value in terms of the current values of 
other variables, bach variable also has a list of related variables 
whose Procedures utilize the present variable as Input. when the 
cumulative change in the value of a variable exceeds a threshold, its 
related variables are activated by adding their procedures to a stack 
of lobs to be run. Thus, if running a process changes the value of 
its associated variable, additional processes may b e activated. 
Execution terminates when the job stack is empty. 

The evaluation process is initiated 
by leading the job stack with the procedures of variables for which 
updated global likelihoods are desired. Initially, the procedures 
of every variable are put on the job stack to obtain global 
equilibrium likelihoods. The consequences of suosequent 
Instantiations are explored by queueing only the Drocedures of 

variables directly dependent on tne values of the instantiated 
variables . 

The above scheme- tor esta^iishina 
global likelihoods is applicable to a variety of constraint 
satisfaction and optimization problems. A set of LISP functions, 
known collectively as the system XDE*ON, has been developed to 











facilitate the creation of network representations for particular 
constraint problems. These functions were documented in Appendix h 
of Kef. 37. for our scene interpretation application, variables are 
used to represent regions, region Interpretations, and relational 
constraints between pairs of regions (e.a., adjacent, above, and the 
like). Region variables evaluate to boundary descriptions, 
interpretation variables evaluate to the current global likelihood of 
tnat interpretation, and constraint variables evaluate to the oegree 
to which argument reaions conform to the stipulated constraint. 

The procedure associated with each 
interpretation variable computes current likelihood as a function of 
local likelihood (i.e,, a likelihood based solely on a region's own 
attributes) and global likelihood (i.e., a likelihood based on the 
current likelihoods of semantically related interpretations and on 
the values of corresponding constraint variables expressing how well 
each relation is satisfied). The variables are interconnected so that 
the devaluation of any region variable (reflecting a Change in 
boundary) will cause all contextual constraints involving that region 
to be reevaluated. Similarly the reevaluation of any interpretation 
or constraint variables will automatically gueue all semantically 
related interpretations for reexamination. The efficiency of tne 
scheme stems from the tact that variables are only reevaluated when 
the value of a contextually related variable changes. 
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in 


Search 


A simple state savlna mechanism has 
been programmed that allows a current computational context to he 
reinstated at a future time. The search context for 4 * consists ot 
the complete network of variables described in Subsection b above 
Plus additional problem dependent var.aoies that typically may 
inclule a global score, a list of previous region instantiations, and 
a priority queue (called IGUEUE) of instantiations yet to be tried in 
that context. The saved context is inserted onto a priority queue ot 
contexts (SOUF.Ufc) to be explored. in general, a search proceeds by 
reinstating the top priority context on SGUEUE, selecting the best 
instantiation from the current iuueue, and then reevaluating the 
network of variables in tne new context created by that 
instantiation. An acceptable solution terminates the search. 
Otherwise, IQUE.UE is updated and the new context added to SQUEUE. The 
search then continues in the top context. 

The nature d the search win oe 
determined D y the problem dependent priority functions used to update 
SQUEUE and ioueue and oy the termination condition. A depth first 
search is obtained by always adding new contexts to the too of 
SOUEUE, a breadth first search is nad by always addinq them to tne 
bottom, and a best first search is realized by queueinq contexts in 
order of global score 165J. Heuristic guidance is introduced into 
the search by the function that updates 1QUEUE. The termination 
















condition can ce chosen to select the hiqhest scorina solution (i.e., 
a complete set of globally consistent instantiations), the first 
solution encountered, or a complete enumeration of all solutions. 

d. Using MSYS 

An Interpretation problem is posed to MSYS in 
the followina way. First, t s e experimenter, using a trackball, 
circles a set of test recions on the displayed Image of a scene, as 
Indicated in Fiqure 49. These regions are retainea on a aisk file. 
Next he enters the constraints to De used in the current experiment. 
He ma' also directly assert sympolic relationships between regions 
(e.q., tnat two regions ne consioered adjacent). This ability was 
useful tor simulating unimpiemented relational procedures and for 
fabricating experimental situations. Interpretation is initiated by 
calling the function Interpret with a region file name or a list of 
regions as an argument. MSYS responds with a complete protocol of 
the interpretation process containing, first, a list of locally 
possible region interpretations and their initial likelihoods, 
second, a trace of all jobs executed from the job list and, third, a 
final list of unique region interpretations with associated 
likelihoods (or else a message announcing failure to find consistent 
interpretations). 

Constraints are entered in the format 
(ADDCOhST Pel Vllst). Pel is a simple relation such as (Above 
Chairback Chalrseat), a boolean expression of simple relations such 


















(PICTURE PICI - 0 3 
(DOOR PIC) - 0 3 
(WALL PIC) - 0 4 


(WALL RWALLI 
(DOOR RWALL) 


(WALL LWAI.L) 
(DOOR LWALL) 


ITTOP TABLE) 


ICBACK CHAIRBACK) O il 
(DOCR CHAIRBACK) 0 202 
(W/ LL LHA IRBACKI 0 688 


(DOOR DR) 0 227 
(WALL DRI - 0 773 


(CSEAT CHAIRSEAT) - 1.0 


(FLOOR FLR) 10 


FIGURE 49 


MANUAL!. v PARTITIONED ROOM SCENE WITH LOCAL BAYESIAN 
INTERPRETATION LIKELIHOODS BASED ON HEIGHT AND SURFACE 
ORIENTATION 




















as (Or (Adjacent Picture Wall) (Adjacent Picture frrame)), or a 
functional constraint sucn as (Homogeneous Door). Independent 
constraints on the same lnteroretation are assumed to be embedded in 
an implicit conjunction. Viist is a list of the interpretations to 
which a constraint applies. Thus (ADDCOWST (Adlacent Picture .vail)) 
(Picture;) requires all pictures to be adjacent to Walls but puts no 
constraint on Walls. If Viist is omitted# MSYS assumes that the 
constraint applies reciprocally to an interprstations mentioned 
within it. 

e. An Example 

The operation of mSyS will be illustrated by 
describing ar. interpretation of the scene partition shown in Figure 
49# using the constraints qiven in Figure 50. These particular 
constraints w e re empirically selected for this room scene domain on 
the basis of their computational simplicity and their suitability for 
use in partially partitioned scenes. Alternative constraints come 

(ADDCONST (QUOTE (NOT* (ADJ DOOR PICTURE] 

(ADDCONST (QUOTE (FUNCTION RQOMPART)) 

(QUOTE (DOOR WALL))) 

(ADDCONST (QUOTE (FUNCTION HOMO)) 

(QUOTE DOOR)} 

(ADDCONST (QUOTE (FUNCTION HOMO)) 

(QUOTE WALL)) 

(ADDCONST (QUOTE (FUNCTION HOMO)) 

(QUOTE CBACK)} 

FIGURE 50 CONSTRAINTS FOR ROOM SCENE INTERPRETATION 













readily to mind: (ADJACENT PICTURE WALL) ana (NUT (ABOVE DuUR 

WALL)), to name two. No formal basis yet exists tor choosinq 
Particular constiaints. 

The first step in analysis is the assignment 
of possible interpretations to the regions based on their local 
attributes. Figure 51 shows the interpretation variables created 
during this initial ’abeling phase ana their associated local 
likelihoods. These local interpretations were obtained using a 
Bayesian classifier, which compared the neiqnt and surface 
orientation of reaions in Figure 49 with those of training regions 
previously outlined in a similar scene.* For each region only those 
interpretations with a likelihood exceeding ten percent of the 
likelihood of the most probable interpretation were retained. Initial 
classification was based on height and orientation, because th'se 
were considered more intrinsic attributes than color and brightness. 
The homogeneity of color and brightness over a given surface, 
however, provided a key global constraint. Note that each horizontal 
surface received a unique interpretation determined by its height, 
but that all vertical surfaces received at least two possible 


•Height and orientation are obtained directly from range data using 
transformations described in Ref. 66. The data used in our 
experiments simulated the output of a developmental time of flight 
laser range finder 137], whose current accuracy is about one inch in 
ten feet. 
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(PICTURE PICI - 0.251 
(DOOR PICI - 00 
(WALL PICI - 0 0 


(WALL RWALLI 0 628 
(DOOR RWALLI - 0 123 


(WALL LWALL) 
(DOOR LWALL) 


(TTOP TABLE! 


ICBACK CHAIRBACK) - O il 
(DOOR CHAIRBACK! - 00 
(WALL CHAIRBACKI 0 0 


(CSEAT CHAIRSEAT) - 1.0 


(FLOOR FLR) 10 


















Interpretations (door/ wall, and when consistent with helqnt 
extremes, wastebasket, chalrback, or picture). Global constraints 
were thus only needea for vertical surfaces. The Hayeslan 
likelihoods associated with the interoretatIons of vertical reqlons 
represent the expected proportion of the Inna je occupied by each 
Interpretation. The likelihoods assigned to (wall Dr) and (Door Dr) 
In Hgure 51, for example, represent the relative areas occupied by 
Wall and Door in the test inane. 

Ihe second step of analysis entails the 
construction nt evaluation functions tor each Interpretation variable 
and the subsequent computation of qlooal equilibrium likelihoods. The 
state ot the data base toileting the attainment 0 £ equilibrium Is 

shb.n in Figure 52. (For conciseness, the unlguelv Interpreted 
horizontal regions have been omitted.) 

Tne evaluation function for each 
interpretation is formulated as a conjunction of local IKelihood 
(see Figure 52) and support for anpiicaoie constraints. The supoort 
for each constraint is expressed as a logical combination of other 
interpretation likelihoods and region relations. Indicating all the 
Possible w a ys that the constraint can pe satisfied in the current 
image. The actual process of creating these evaluation functions Is 


described In Hef. 5H. 








VARIABLE| (DOOR RWAIL) 

VALUE| .133 

PROCrOURti 

lA^C# ,227 (AMD* (NOT* (OB* (D30P DR) 

(DOOR PIC) 

(DOOR CHAIRBACK))) 
(NOT* (AND* (ADJ R»AI.L PIC) 
(PICTURE PIC) 

RELATIVES! 

((OPTION (u'JCP RNALl) (WAIL RWALL))) 

( ( Af'P* (ACU P * ALL PIC) (POOP P'ALL))) 

((OP* (DC'PP L*ALL) (OUUP RaALII)) 


VARIABLE | (VALL P/'AI l ) 

VALUE« .P?B 

PROCEDURE! 

(AND* .773 (> 0 T * (OP* (*»LI. OR) 

(WALL PIC) 

(WAbb CRAIRPACK) 

RELATIVES! 

((LPTIO.” l I'L'OR P“ ALL) (WALL HP AIL 1 ) ) 
((up* (mALI- La ALL) ( *ALL P'ALL))) 
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VARIABLE! (DOOR L ' ALL) 

VALUEl .1/J 

procedure: 

(At L* .227 (A'-D* (NOT* (Op* (POOR DR) 

((.OCR PIC) 

("DOR CRATURAC<))1 
(VOT* (A-O* (&r.,j L<ALL FIC) 
(PICTURE Fir) 

RELATIVES! 

( ('.’FT I ON (POOP LaALL) (VAIL V* ALL) ) ) 

((AM>* (ID J LaALI FIC) (DOuP LaALL))) 

({OP* (LCPh b 'ALL ) (DOOR P a A L! ) ) ) 


VARIABLE.! ( - Ab1 I a Al l 7 
VALUFl .B7P 

PROCEDURE! 

tAUD* ,773 (’Or* MR* (WAM OF) 

(WALL P1C1 
(aALL CRAIBBACK) 



RELATIVES! 

((UlTIGw (POOH LaALL) (WALL IP ALL) )) 
((OP* fVAtL LWA.LL) ('ALL R'-'ALL! ) ) 




VARIABLE: (P0CP C-AIK-AC*) 

VALUE: 0,0 

PRClCEDul-k : 

frr:c*K Cf-AJPPACK) 

RELATIVE.*: 

((OPTI'll. (POOR CHAIRBACK) (CsaCW C^AIR^ACK) (hALL Cri&lRftAC<))) 
(fOA* (Pr(.p r.R) (POOR PIC ) f?00P COUPAAC*))) 


VAHlABl E; (CRACK OAlHPACW) 

VALUf I .11 
PROCEPl'Ht : 

ffPACA CLAIRFaCA) 

RELATIVES: 

( ( ‘ l f-Tl r ' t > f r -OOR CHaIHHACK) c c ^ a C b CHAINnACM f/M T. CRAIRPAC*))) 

VARIA-HE: ( -A lL C«A]RRAC<) 

VALUE: o. ? 

PROCEDURE: 

(vA!.r. Cmaif-pack) 

RELATIVES: 

r (L’PTI.k: (Hi'-R CiAlMACK) 'C >4C C-iAls-tr^j r,.Al,L CRAI^ac* ) ) ) 
r CI * (•AM I'H) (WALL T C 5 (..A.,L •> A t r f »C M )) 

VARIABLE: (HClUFt t IC J 
VALUEl .251 

PROCEPURt: 

tA"L# ,i f-C'T* f-F* (A NO» (A.1J P VALL FICl 

(P'T-t h-Atr,)l 
(A•.'!>• fAU'J L-.At.i FICl 
(COCO L--ALL1 

RELATIVES: 

( (Of-Tirr. (PICTURE PTC) (OCVR PIC) ( *’»! I, t»IC))) 

((»M>* (At'J F*ALt. »iC) (PICTURE FTC))) 

( f A’-D# (APU L*AIL PIC) (PICTURE PTC))) x 

VARIABLE: (L'OC'P PIC) 

VALUE: 0.0 

PPOCF DURE: 

(inf.R (TC) 

RELATIVES: 

((OPTIC'! (PICTURE PIC) (Di’JP PIC) (WALL PIC))) 

((OR* troop up) (onnR pic) (coop cbairbacm)) 

VARIABLE: (WALL PIC) 

VALUE: 0,0 

PROCEDURE: 

(WALL PIC) 

RELATIVES: 

((OPTION (PICTURE PIC) (OOUR PIC) (WALL PIC))) 

((OR* (WALL DR) (WALL PIC) (WALL CWAIRBACK))) 

FIGURE 52 STATE OF DATA BASE BEFORE SEARCH (Continued) 
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VARIABLE| 
VALUE I 
PROCEDURE 


RELATIVESi 


VARIABLE| 
VALUE| 
PROCEDURE! 


RELATIVESI 


FIGURE 52 


DOOF. DR) 

.175 

(AND* ,227 (AM)* {NOT* (OR* {DOOR V* ALL) 

(DOOR RSALL) ) ) 
(SOT* 0.0))) 

({OPTION (DOOR DR) (aALL DP))) 

{(OH* (DOOR DP) (OOOP PIC) (DOOR CHAIRHACK))) 


’"ALL DP) 

.142 

(ADD* .773 ( S' 0 T * (OP* ("AlL L»ALL) 

('/ALL 3"ALL) 

((OPTION (DOOR DP) (WALL DR))) 

((OP* ("ALL DP) { f> ALL PIC) ( •JALL C* A IRR AC* ))) 


STATE OF DATA BASE BEFORE SEARCH (Concluded) 
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fhe constraint (Homogeneous Wail), requires 
that an regions interpreted as -/an pave similar brigntness, mis 
constraint Introduces a clause of the form (nut* (uh* (*ALL 
Rl 1) ---(Wall Rin))) into the likelihood function of each wall 
interpretation (Wall Hi). The disjunction includes all reqions 
admitting the interpretation wall# wpose nrigntness differs from that 
of Hi by more than ten percent. The effect is to reduce the 
likelihood that region Hi is wall by an amount proDortlonai to tne 
likelihood that these other# nonhomooeneous regions are thouqht to be 
wall. Homogeneity constraints on other interpretations nave analogous 
effects. 

Ihe constraint, Poompartition, requires that 
the brightness of all reqions admitting the interpretations Wall or 
Door (1.e,, surfaces that partition rooms) be similar to that found 
at the very top of the image vertically above their centers of 
gravity. Region interpretations failing this test are rejected 
outright by pinning their likelihood to zero. The likelihood of 
interpretations that pass is unaffected. The Roompartition 
constraint is closely related to the previously discussed 
homogenerlty constraint and was included to handle cases where that 
constraint was ineffectual because tne experimenter failed to circle 
enough regions. It is based on an assumption that, in a standard 
view of a room scene, wall or door win appear at the top of the 
image, and moreover, that walls and doors never appear vertically 
above each other. The effect of this constraint was to eliminate 

















’Door" and "Wail" as DossiDie Interpretations of vertically oriented 
regions lacking appropriate vertical extent. Specifically "Door" ard 
"*ail" w e re eliminated as possioie Interpretations of the reaions 
Chalrback# Pic# and wbskt# leaving those regions with unique 
Interpret at ions . 


The relational constraint# (NOT* (Adjacent 
Picture Door))# is self-explanatory. It Introduces a clause 
reducing the likelihood that a region Is a Picture ty an amount 
proportional to the likelihood that adjacent regions are thought to 
be Doors. A loose definition of adjacency nas been adopted In order 
to utilize this constraint in a Partially segmented scene: two 
regions are adjacent If the line connecting their centers does not 
pass through a third region. 

The Numerical equilibrium likelihoods shown 
In Figure 52 w e re obtained by executing the evaluation functions as 
If they were Bayesian combinations of Independent probabilities 
(l.e.» a conjunction of likelihood values evaluates to the proauct of 
those llkellhods# the negation of a likelihood to one minus the 
likelihood# and a disjunction of likellnoods to one minus a product 
of the negations of the likelihoods). More sophisticated 
quant iflcatIon schemes are given In Ref. 5tf, 

The final stage of analysis entailed 


searching for a set of unique Interpretations that were globally 

















consistent. The only ambiauity remaininq in the global equilibrium 
context involved the regions Dr, L.waii, and RWali, all of which still 
aimltted both door and wall as possible Interpretations. Homogeneity 
constraints forced LWall and R w a 11 # both light colored reqions, to 
take the same interpretation (either wall or Door) and Dr, a dark 
"olored region, to take tne opposite interpretation. this basic 
ambiguity was resolved during the course of searching by the 
adjacency constraint on pictures, which led to a contradiction when 
either LWall or RWall was instantiated to Door. 


The search oroceded without any backup 
because the correct interpretation of each region already had hlqher 
likelihood than any alternative interpretations in the initial global 
context (Figure 52), The relative likelihoods of tnese correct 
interpretations increased monotonlcaliy with each successive correct 
instantiation. The final interpretations for the regions in Flqure 
49 are presented in Fig re S3. A detailed trace of the reasoning, 
showing all instantiations and resulting reevaluations, appears in 
Ref. 58. 


Conclusions 


A working implementation of a new seen? 
interpretation technique has been demonstrated. So far, the 
experimental results are inconclusive because of Ihe simplicity of 
the test scene and the reliance on simulated range data for assigning 
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(PICTURE PIC) 10 
(DOOR PIC) - 0.0 
(WALL PIC) - 00 


(WALL RWALLI 
(DOOR RWALLI 


(WALL LWAl LI 
(DOOR LWALLI 


•TTOP TABLEI 


(CBACK CHAIRBACKI - 10 
(DOOR CHAiRBACK) 00 
IWmLL CHAIRBACK) 00 


(DOOR DR) 10 
IWALL DRI - 00 


ICSEAT CHAIRSEAT) - 1,0 


(FLOOR FLR) - 10 




















initial interpretations. The experiment win be repeated shortly 
usinq the actual laser range finder. Expectations are that the data 
will provide height estimates accurate to a few incnes but that it 
will be to.*/ noisy to yield usaole measures of local orientation. 
Consequently, additional constraints win be needed to distinguish 
horizontal and vertical surfaces at the same height, such as taoletop 
and wall. It would also be interesting to see how far one could qet 
without direct range data by infering height and orientation entirely 
from pictorial clues such as image, height, and shadows. 

Scene interpretation typifies a class of 
problem solving tasks involving large numbers of globally interacting 
constraints. A general constraint optimization algorithm is presented 
tnat, for sufficiently constrained proolems, is more powerful than 
conventional A1 search methods. This algorithm is implemented by 
representing interpretations and constraints as (simulated) 
asynchronous parallel processes. The resulting system organization 
features efficient data driver control and knowledge sharing. 
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IV. 


SUPPORTING WORK 


A. Introduction 

A large amount of ancillary eftoit Is required to support a 
project of this scope. Much of the programming In the project is 
done in our special AI language GLISP. (See Ref. 20 for a general 
discussion of some of the new Al languages.) Several additions to the 
power of QLISP as well as routine maintenance have been performed py 

B. M. Miner and Daniel Sagalowicz. These are described in Section 
IV.B below. 

In Section IV,C we discuss supporting hardware work, Bert 
Meyer has been responsible for the development of A/D a n d D/A 
interface equipment and programs connecting CBC gear to a PDP/U 
computer and the PDP /11 in turn to our main PDP/io. A. fc. Brain has 
continued his *ork on our scanning laser range finder. 

Finally, in Section IV.D, we present a short summary of the 
current status of the SRI Artificial Intelligence Center computer 


system. 




















a, uL I £>p 

The Artificial Intelligence Center uses QLISP 118# 20# 3b, 
67] in a variety ot applications and makes it available to users 
outside the AiC tor other purposes, QLISP extends INTtRLISP L t» 8 j to 
provide powerful features for use in artificial Intelligence 
applications. It provides a context mechanism for hypothetical 
reasoning and access to Planning spaces; it provides automatic 
backtracking so tnat plannina can pinch off fruitless considerations; 
it provides pattern-directed function invocation to simplify the 
programming of case breakdown? and it provides D a ttern-directed data 
storage and retrieval to permit symbolic access to the data base at a 
high level of abstraction, 

Effort spent on QLISP has reguired both intensive work to 
improve QLJSP's behavior within its scope and extensive effort toward 
extending it. The interval of this report includes three major 
accomplishments, ULISP and the INTERLISP compiler have been made to 
understand one another# permitting a substantial speedup of programs 
using the system. The pattern matcher has been replaced with a 
unification program providina enhanced Power and flexibility at a 
reduction in storaae space and computation time. we have designed 
and partlaJly implemented a replacement for the data storage and 
retrieval mechanism# which we hope will provide both true associative 
access and allow graceful extension to much larger data bases than 
can be stored in any less drastic extension ot the current data 














storage system. Finally, since GLISP is continually developing and 
responding to new needs, w e hav e an ongolnq proqram of fixing new ana 
old buqs and providing miscellaneous other extensions. All four 
areas of concentration have contributed greatly to the utility 0 f the 

system, while only the second two of these areas nave noticeable 
extensive components, 

A programming system can be more or less tractable depending 
on the cleanliness of the face presented the user. w e have maintained 
a consistent effort to ensure that QLISP "looks reasonable" t 0 both 
naive and sophisticated users. At one extreme, for example, QL1SP is 
almost completely invisible to the us e r (or program) using none of 
its special features. Of greater interest, however, is the face 
QLISP presents to a us e r ( 0 r program) using its features. The 
extreme flexibility of INTERLISP has permitted us to build a nighlv 
automated Interface between it and GLISP. a prime example of our 
efforts to maintain a clean user interface appears in Subsection 
IV.B.1 just below but a few points may oe briefly made here. Even at 
tne TENLX level, access to GLISP is no more difficult than access to 
LISP * GLISP will not intrude on a user who Ignores it, but the 
user can freely intermix LISP and GLISP with no need to specially 
declare the presence or absence of GLISP in any segment of the 
program in question; GLISP system routines win be called 
automatically When appropriate and at almost no other vime. 
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The rest of this section discusses the work we have out into 
QLISP since the beginning of 1974. Each major aspect of this work is 
treated in a separate subsection# aitnouqh the reader should 
recoqnize that there is inevitably some Jeqree of interaction and 
overlap between the contents of tne various subsections. The 
"staff" of the ULISP effort has turned ov e r completely during this 
interval# with Earl Sacerdotl and Rene Reboh turninq their efforts 
elsewhere# while Michael *ilber and Daniel Sagaiowicz now do most of 
the work. The current level of effort is almost two full-time 
People# but w e expect this to diminish considerably as tne current 
period of intensive development ends. 

1. Compllinq User Proqrams 

One of the qreat shortcomings of QLISP was overcome 
toward the end of 1974: users could not compile a proqram conta<ninq 
any QLISP. Because of this conspicuous lack# QLISP users tended to 
use QLISP only tor developing their algorithms and then# just for the 
sake of speed# recode their routines at no small expense of their 
time to use LISP where it could be made to do just as well. We have 
Interfaced QLISP to the compiler in a manner about to be described v 
The resultant speedup of user programs nas made QLISP tractable to 
many applications in wnich People formerly programmed around QLISP 
either fcy that kind of reprogramming or by avoiding its use 
altogether. The QLISP system is heavily enough used that we can 
presume the speedup to have enhanced the overall performance of tne 
computer system with the result of making more computer time 
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available to all, but the number of variables precludes our having 
anv good statistics with wnich to back up this conjecture. However, 
this improvement alone has helped make uMSP the standard programming 
system for a large part of the CBC system. 

ULISP was extended to compile user code by redefining 
two ot the interior functions of the compiler. In one case, the 
modification was inessential in concept; we simply provided automatic 
tracing of compiled functions as a parallel to the automatic tracing 
provider for any QLAMBDA function defined by any other means. 

The other, however, was essential for this extended 
notion of compilation to work at all; it entailed the translation of 
any QLISP constructs to LISP code that would call routines internal 
to the QLISP system, first ot all, the structure of LISP'S comoiier 
required the translation of QLAMBDA functions to the corresponding 
LISP code as a special case. The body of the function is translated 
in almost the same manner as during evaluation of an interpreted 
function containing QLISP, When QLISP code is interpreted, we can 
(and in tact prefer to) translate only those parts of the function 
actually being used. This avoids unnecessary expenditures of time 
to translate possibly large amounts of code that may never get used 
before the translation must be discarded for any of a number of 
reasons, on the other hand, when a function is compiled, its entire 
body must be translated in order that the compiler generate the 
proper calls into QLISP. Finally, a small number of flags is used 





















to determine whether to take the time to translate the function at 
all; some flags are set by the system, and some are provided for the 
user to Indicate the presence or absence of QLISP in specific 
functions. 

Ti ere was a bit of a challenge to be met in 
maintaining the compiler's user interface after adding the QLISP 
compatibility. In order that users need not become experts in the 
inner machinations of INTERLISP or of QLISP, we needed to let them 
compile, redefine, edit, and run individual functions or files of 
functions in their interpreted or compiled forms in any combination 
made available by 1NTEHL1SP without any need to make any special 
allowances for the possible or known presence or absence of ULISP 
anywhere in the code in question. On tne other hand, this 
flexibility should not be provided at a cost in the overall system 
performance; speed bears just as heavily on the appeal of a system to 
users as do flexibility and unobtrusiveness. 

Unfortunately, we could not simultaneously provide 
speed and unobtrusiveness in tne case of the compiler interface. The 
compiler has provisions to automatically call a function supplied by 
the user (i.e,, QLISP) for most of the Places where QLISP is likely 
to occur, but unfortunately, that mecnanism wm not handle all tne 
cases a user could generate. In this case, we were faced with no 
alternative but to intercept every function before it wa compiled 
and make sure all QLISP constructs were translated. This process 





















approximately doubles the time taken to compile any qlven function. 
This is unavoidable if QLISP is actually present in the function 
being compiled, but it is entirely gratuitous it the function uses 
only LISP. we solve this Problem (to the extent it is soluble) ny 
providing the system of flags alluded to previously. The default 
state is that QLISP is suspected to be everywnere; we assume that the 
reason the user is using uLISP at all is that he has QLISP somewhere 
in his code, and as long as the user does not tell us where, we must 
look everywhere. The user can of course cnanqe the default state, 
but it is probably more apropos to indicate lust where to look to:- 
QLISP; to this end, we provide more flags to indicate the presence or 
aosence of QLISP in a whole tile or in specific functions in a file, 
with each indication overidlng the less precise ones. 


Finally there was the problem of preserving the 
tremendous flexibility of 1'JTERLISP, which permits the user to load 
(or refrain from loading) whatever combination of compiled and 
interpreted functions or files of functions that might be appropriate 
to production runs, debugging, compilation, development, or massively 
altering the QLISP Program, This one was easy* it just required 
tha we realize what needed to be done and exercise sufficient 
cleverness In doinq It. As it turned out, the implementation was not 
difficult. All w e needed to do w a s install a routine in the standard 
INTLFLISP file package to put the symbolic file into the proper 
format before it was written out. This routine makes slight 
alterations in the format of the file from the foimat the user may 













have specified and# of course# It can cope with almost anything the 
user can specify# especially the undoinq of its changes from a 
previous generation of the file. 

Ihe reader unfamiliar with INTFrLJSP should realize 
that we had to spend relatively little time in actually performing 
this surgery on the lNlt'KiTSP compiler or file-malntenance package. 
All the access points we needed were available to us. Some were 
available as features deliberately provided tor applications such as 
ours. For example# the translation can be forced by setting a flaq 
for the compiler# and it wm automat ica 1 ly call DWiMUSERKN on 
encountering something it does not recognize; anybody can define new 
commands for the file-ma1ntenance packaqe simply by putting an 
appropriate entry on PRETTYDbfMACROS, Utner access points are 
available simply because of the structure of INTFRLISP; we get a 
first look at a function being compiled or at a symbolic tile beino 
written by the artifice of moving the definitions of certain critical 
functions to other places and installing in their stead our own 
functions# which do the preprocessing before calling the saved 
definitions. 

2. Unifying Pattern Matcher 

At the end of 1974# the pattern matcher in QulSP was 
replaced by a unification proyram, which increased the power 
available to us in the pattern matching operation. The unification 


















principle is explained in IbdJ and 170], Wnen appropriate, we now 
Jnlfy two expressions rather than patching a pattern to an 
expression, The distinction is one of symmetry. With unification# 
any variable in either expression is free to take a value, and all 
variables with a given name must take the same value in noth 
expressions for the match to succeed. (Formerly, values could be 
assigned only to variables occurring in the expression designated the 
"pattern".) For example, the pattern (CONNECTED pump -X) will not 
match the expression (CONNECTED *- Y PLATFORM) because the variable X 
can take no value to make the pattern match the target expression. 
However, if the pattern is just taken as another expression to be 
unified with the target expression, the two expressions wm unify, 
with SX and $Y taking the values PLATFORM and PUMP, respectively. 

The unification program is far easier to maintain 
than the old pattern matcher, and preliminary measurements indicate 
that despite its greater Power, it is a more compact and sllgntiy 
faster program. While this change is too recent to allow us to have 
exploited it extensively, the next few paragraphs outline some of the 
uses to which we feel it could be put. Readers familiar with QLISP 
should note that in this discussion of the unifying pattern matcher, 
we win use the term "variable" in a estricted sense: since we are 
concerned with the assignment of values to variables to the exclusion 
of any concern with the instantiation of a variable to retrieve its 
value, we will use the term to mean only those variables that are 
expected to take values (and are written like-X in Ol,ISP), as 
















opposed to those that are expected to alreadv have values (and are 
written like $X). 

lhe symmetry character 1stlc ot Pattern matching In 
the new regime can he used In several ways. perhaps the most 
oovious Is to construct expressions using variables supplied oy the 
program in conjunction with Items supplied to the program as data. 
For example, to tlnd the difference between (Class a B C) and (CLASS 
A B D), one need only unify classes nuilt by adding distinct 
variables to the two classes. Then each ot the two variables will 
take for Its value the extra item in the other set, as is necessary 
for the two constructed classes to unify. if the two classes above 
are supplied as the values of Si and $J, then MATCH (CLASS $$I --X) 
(CLASS $$J *-*-¥)) will store D and C as the values of $X and Si, 
respectively. 

Our new symmetry qives us a facility lacklnq in LISP 
(and heretofore also in QL1SP) due to the traditional conception of 
LISP as e mechanization of a formal logical theory. Uriqinaliy LISP 
nad the notion, as did most formal mathematical theories, that the 
evaluation ot a function generate precisely one result; if several 
answers are to be returned, they are all coded into the single 
result, sometimes givina schemes rivaling Goeael numbers in 
obscurity. With the emerging concept of LISP as a programming 
system, functions would sometimes pass their results back through 
free variables whose names were either given as Dart of thp 











specification of the function or (awkwardly) nassea as arguments, mow 
that QLlSk variables in the expression supplied a ULAMBOA function 
can be assigned values as a result of the oattern match on entering 
the function# parts of the 0LAMBDA pattern can be given as values to 
those variables during the match. A more interesting case arises 
wnen a variable in the supplied expression is matched to a variable 
in che OLAMBDA pattern. Then, when the pattern's variable is asslqned 
a value in the body of the QLAMBDA function, the value is pass e o back 
to the supplied variable, giving QLISP a facility analogous to the 
"call by address" concept of the Algol thinkers. 

The reader familiar with QLISP may have noticed a 
slight problem that arises, however, in precisely the case that an 
expression containing Variables is supplied as data to a ULAMRDA 
function. The crux of the problem is that a qlamrda implicitly 
declares a new local context for any variables occurring in its 
pattern when the pattern is matched to tne data expression supplied 
It. Since variaoles that might occur in the data are Independent of 
variables in the program, even though they may nave the same names, 
we should regard them as unrelated. Of course, the way we handle 
that case is to enter the (new) Pattern matcher with an argument 
specifying that it is performing a olambda rratch. men the 
unitler--i.e., the (new) pattern matcner--Decomes asymmetric in the 
sense that any variables in the "data" expression are regarded as 
distinct from any variaoles occurring in the "pattern" expression 
that may have the same names and receive their values in the context 











outside the body of the QLAMBDA function. This procedure Is known as 
"standardizing variables apart" in more formal discussions of tne 
unification principle. 

Unification can also ce used to aet the effect of 
storinq universally quantified statements in the data base. It an 
expression with variables is asserted and then a query with variables 
is performed, and if the query can successfully match the asserted 
expression, then the result of the match may restrict the query in 
such a way that it gains information. For example, if cne asserts 
that for every X, X+l is greater than X and then later one has a 
symbol A and desires something greater than it, the prober query wm 
produce A+l; in QL1SP, it looks like this: after performing the 
assertion (ASSENT (GT (PLUS*~x i)«-X)) and then later the qu-»ry (IS 
(GT <- Y A) UNIFYING), then (FLUS A 1) win be stored as the value of 
$Y. Such a query presents no problem to the unification program, of 
course, but the access to the data base must be much more extensive 
to allow for such a case because everything in the data base must be 
retrieved it it has variables in it that miqht let it unify with the 
query. (The more usual data base access is far cheaper because the 
variables in the query must be matched to constants in a data base 
item for the item to be retrieved.) Thus, because of the relative 
expense and infrequency of this kind of query, we require the 
inclusion of the wore UNIFYING in the query statement. 






















The user interface consiaerations for the unifyinq 
pattern matcher are somewhat different from those for the compiler. 

Speed, flexibility, and noninterference are the criteria here too, 
out at least w e do not nave to allow tor reasonable behavior in the 
face of as wide a range of user behavior. Sucn a consideration has 
again shown up: the implied generality includes a v e ry expensive 

case of relatively little interest, and so we require the user to 
specifically indicate the appropriateness of the "bad" case. Here, 
we prefer that the general case be the default: however, the marginal 
cost is so great tnat the marginal utility is very small. Therefore, 
*e specify the defaults here to be just the opposite of what they are 
in the case of the compiler. Tnere tne cost is a mere factor of 
two in an operation (i.e., compiling) that is not done too often, and 
the benefit is that everything will be nandled correctly, Dn the 
other hand, the general handling of this "bad" case would 
significantly Increase the expense of retrievals from the data base, 
a frequent operation, and user programs would usually realize no 
benefit. Of course, another aspect of tne face QLTSP presents to 
its user Is exemplified by the tact that these modes of ooeration are 
Duilt into the system only as Its default choices for the case that 
the user does not indicate a preferred mode of operation. This is 
why w e can be so apparently cavalier about a factor of two; if the 
guess causing it is incorrect, the user can easily correct it. 
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Data storage 


As another part of the QLISP development* *e are 
currently workino on a new data base organization. In the next 
paragraphs we are going to present successively the old data base 
organization* called the Discrimination Met* and its possible 
deficiencies; the ideas behind the new organization and its possible 
advantaaes; and finally some of the deeper details of the new 
organization. This new data base organization is in a stage of 
development and about three man-montns have been spent on it, 
consequently* the details are still sketcny and it is still too early 
to say whether the justifications presented in the next paragraphs 
aie confirmed by experience or whether some major problems are golna 
to appear that cannot De overcome. 

a. Tne Discrimination Net--Oraanizatlon and 
Discussion 

The discrimination net organization is based 
on what is known in the literature as the "trie" technique 171—731• 
Our own implementation w a s devised by Jeff Ruiitson and qreatlv 
optimized by Hene Reboh. Altnouqn it is not the purpose cf this 
report to present the details of tne disci j.mlnat ion net* we think it 
would be useful to explain it in general terms with a few very 
trivial examples. Let us suppose that a user executes the following 
QlISF-llke statement: (ASSERT(CONNECTED PUMP PLAT FORM)ln-context 
















Cl). This inserts the record Hi: <NErEXPRESS10W(CUNNLCTED pump 
PLATFURMHCUMEXT C 1 (MODEL VALUE TRUE ) J > Into the data base. Since 
there is only one record in the data base, the organization tree 
(also known as the discrimination tree) has only one node that points 
to the (unique) record Hi. Then the user executes the following 
QLISP-Iike statement: (ASSERT(DISCONNECTED PUMP PLAIKUHM)in-context 
C 2), Then the data base qets a second record: R?: 
{NETEXPRLSSI0N(D1$C0NNECTLD pUMp PLA THORN') ICON TEXT C2 ( MODELVALUE 
THUt )J} now the discrimination tree has two terminal nodes , one 
Pointing to Hi and one pointing to R2. Hut to enable the retrieval 
operations to distinguish easily between these two nodes a third one 
is created; it is only a discrimination node with Just the 
information necessary to distinguish Hi and R2. In this example the 
needed information is that the two records differ by the first 
element, which in the first case is CONNECTED and in the second case 
it is DISCONNECTEO--as indicated in Figure 54. 
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FIGURE 54 A DISCRIMINATION TREE WITH TWO NODES 
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If now a third assertion is executed such as 
(ASSfcKT(DISCONNECTED PUMP PUMP-PULLEY)in-conteXt Cl)# rhen a third 
record is created, and the discrimination tree now has three terminal 
nodes and two discrimination nodes# ?s follows. The first node is as 
before and distinguishes between R1 and the rest, the second one 
distinguishes between R2 and R3. Figure 55 snows the result and is 
self-explanatory, 
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FIGURE 55 THE SAME DISCRIMINATION TREE WITH A THIRD NODE ADDED 

Already at this point a number of remarks may 
be made without golnq any deeper into the details of the 
Discrimination Net Implementation. First# each record Is in fact 
composed of two Parts: Its "name" proper (e.q.» (CONNECTED PUMP 

PLATFORM)) and a list of triples of the form (context-name 
property-name property-vaiue) (e.g,# (Cl MODELVALUE TRUE)), It is 





























important to notice that the discrimination tree itself, which Is 
used tor data retrievals, does not include any information apout the 
properties. This is due to the basic philosophy of the 
discrimination tree, and such information cannot be added to it in 
any w a y: once a record is attached to a terminal node it remains 
attached to that node no matter how its properties are modified, said 
in other words, the discrimination is only based on syntactic 
infox"stion and not at all on semantic contents. The testing of the 
property values win still be done, but as an "after-pass"; i.e., 
after all the records that Pass the discrimination testing have been 
assembled together. This may correspond to a waste of time, which 
could theoretically be avoided with a diffe.ent scheme it that scheme 
were to use the semantic information to aid in that discrimination. 
In other words, we are looking for a scheme that keeps both semantic 
and syntactic information in the structure that is goinq to be used 
in the data retrievals. 


A second important remark is that the 
semantic contents of the data base Dears very little Influence on the 
way the discrimination tree develops itself, it is even amusing (if 
not very profound) to notice that for the same records stored in the 
data base, the organization tree may be very different depending on 
the order in which these records were stored. Une of the 
consequences of this tact is that the amount of work needed to do a 
retrieval is more or less independant of the semantic contents of the 
data base, but depends heavily on the number of nodes in the tree, or 
















more precisely on tne width and depth of the tree. To put it 
differently, after a long data base development, It can he expected 
that on the average, every retrieval will be long, since It will have 
to go through a great number of discrimination nodes, and this 
complexity will be more or less independent of the number of 
reasonable candidate records resulting from a given retrieval 
request. This last point Is Important, since it is generally felt 
that a good retrieval scheme should have an expected retrieval time 
roughly propo ‘tional to the size of tne retrieval result. 

The tnird criticism of the Discrimination Net 
is also related to the lack of semantic information in the 
organization of the discrimination tree. Because of this lack, 
there Is no w a y for the language to decide how the data base should 
be organized on peripheral mass storage files, when we need to 
overlay the data out of the main memory. This task will somehow have 
to be left to the users. Concerning this last Point two remarks 
should be made. First, in tne environment in which QLISP has been 
known to operate In the past, i.e., the research environment, a 
strong case may be made that most users do not know how their data 
should be organized on peripheral flies In order for their programs 
to be efficient. Moreover, even if each of them knew, the tact that 
the same data base is used by various users of different needs would 
necessitate that these users get together and decide collectively how 
to organise the data base; the result would be a division of the data 
Into a great number of small ''buckets" in which the records are 









stronqiy related In a semantic sense. it Is our contention that 
such a subdivision is goino to be difficult to create unless it is 
done automatically by the data base storaqe mechanism, in our case 
QLISF itself. The Point is tnat the Discrimination Net does not qiv e 
QL1S? any help whatsoever in that direction. 


It would De unfair not to mention at this 
point some of the possible advantages of the Discrimination Net. The 
main one is that it works and has been workinq satisfactorily, if not 
efficiently, tor the past few years. The second one is that it 
probably requires less storaqe than anything else we could devise. 
And finally, all the disadvantages mentioned above are subject to 
strong discussion and will have to be measured in comparison with 
whatever other scheme we may implement. 

b * Proposal for a New Data Base Organization 

After an extensive search of the literature, 
it appeared to us that only two types of data base organization could 
be attractive for UL1£>P to use; one is the trie technique, of wnich 
the discrimination net is one example; the other is the inverted 
index technique. Essentially all the trie techniques would have the 
same types of disadvantages as the Discrimination Net, and many of 
them would not be attractive for QL1SP. Thus it was logical to try 
the other possible type, 
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Let us quickly explain Dy an example now the 


inverted index works. The records stored in the data base are the 
same as in our example used for tne Discrlmlnation Net. The first 
record HI is stored in the same way, but in the data base we add a 
new structure called the Inverted index which has a number of nodes 
called CONNECTED, PUMP, PLATFORM, CUNTEXT-C1, MfJDELV ALUL, and TRUE. 
Under each of these nodes we out a pointer to PI, when the second 
record H2 is added we create two new nodes in the inverted index 
called DISCONNECTED and CONTEXT-C2. Moreover, *e put a pointer to H2 
under the nodes DISCONNECTED, PUMP, PLATFORM, COMTtXT-Cl, MODELVALUE, 
and TRUE. Now some of the nodes have two pointers , some nave only 
one. After the third record Is added, the Inverted Index looks like 
this s 


CONNECTED; Rt 
DISCONNECTED: R2 R3 
PUMP: R1 H2 R3 
PLATFORM; PI R2 
PUMP-PULLEY: R3 
C0NTEXT-C1: R1 R3 
CONTEXT-C 2: R2 
MODELVALUE: R1 R2 P3 

TRUE; R1 R2 R3 

We keep building the data base in this way, 
When a retrieval is attempted the algorithm is quite simple, at least 
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in theory: if we are locking for every record that mentions tne 
PLAipnK.i, we look in the inverted index for the node PLATFORM and 
find these records immediately. If we look for all objects connected 
to the PLATFORM in context C2, we take the intersection of the nones: 
PLATFORM, MOOELVALUE, TRUE, and CONNECTED. All this is Very simple 
and quite well known# but it has one major inconvenience. Taking the 
intersection of olq sets is a long operation# and the lenath of that 
operation is not related to the size of tne result but to the size of 
the sets with which one started. This is considered to be a major 
disadvantage of such a technique. In our example, to find all the 
objects that are connected to the PUMP-PULLEY in context Cl# we would 
take the intersection between five nodes (CONNECTED, PUMP-PULLEY, 
CUNlExT-ci# MODELVALUE# and True) to find out that there is no such 
object; this is a very trouoiesome loss of time. 

To alleviate this problem, we can use the 
technique of subdividing the indices by forming some of the 
Interactions when data are storea. Then, the index at any given node 
may itself be indexed by some other index nodes before yielding a 
list of records, in our example, the index for DISCONNECTED could be 
subdivided, with the next level of indexing distinguishing CONTEXT-Cl 
from CGMTEXT-C2, The result would be that under each index the list 
of records would be shorter and the intersections easier to do. The 
Indices that have been subdivided would point to their subdivisions. 
The question then is: which indices should be subdivided? If they 
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are subdivided to the point where they cannot he subdivided anymore, 
then we would create a hune number of indices and the inverted index 
structure would be bigqer tnan the rest ot the data base, which 
clearly would be inefficient. Un the other hand, if we subdivide 
them only to a predefined depth, then there win always be some 
indices on which the intersections would be too costly. 

In the implementation we are thinking about,, 
we have solved this problem in the followina way. Under eacn index 
we keep account ot the number of records it nolnts to; when this 
number becomes (.00 large--!.e., biqger than a given threshold (which 
is user moditiable)--then the index is subdivided. In this w a y we 
guarantee that no Intersection will occur over sets that have a size 
bigger than the threshold. Of course we also give the user the 
Possibility to declare that some Indices must always be subdivided or 
must never be subdivided. 


The influence of the threshold on the way the 
data base operations behave is not a simple one. When the threshold 
is small, the number ^t indices becomes large and tnerefore the 
Inverted index list becomes large; also the time taken by the 
Intersections becomes small, but It Becomes more difficult to get to 
the point where the actual Intersection occurs, Un the other hand, 
the data base operations that end up taking unions of sets of records 
definitely become longer. it therefore will oe necessary to measure 


whether the advantage of faster intersections outweighs the other 












disadvantages. It may be that tor a given program there is an 
optimal threshold, and the difficulty win be to find what it is. 


In this new data base organization the user 
has the choice of the "subdivision threshold" as we have just seen 
above. However, this Is not the only freedom ne has. A t the 
beginning of this subsection, we saw that the "basic indices" are the 
elements of the record name, the context, the property names, and 
values. m tact, this does not need to be so. The user win be 
given the choice of deciding what are his basic indices. m some 
cases, ne may decide that a property name or v a iu e is not needed as a 
basic index, mainly because no associative retrieval will ever be 
done on it, or that it will be so rare that the disadvantages of 
having it as a basic index outweighs its advantages. 


At this point we have given the general iaeas 
of our new implementation and win leave some of the more intricate 
details for the next subsection. It is now time to explain why we 
feel that we m a y have solved some or all of the problems w e think 
occur with the Discrimination Net. The main point is that now the 
"inverted list" Includes both syntactic and semantic information. All 
of these may be used during associative retrieval and no additional 
pass is necessary for the semantic information testing alone. 

The second point is that now the retrieval 
time is roughly proportloral to the size 


of the result of the 















associative retrieval which, as mentioned earlier, is a sought-after 
characteristic of "good" associative retrieval techniques. with 
respect to this point, it should De noted, though, that we do not 
<now at this time whether the associative retrieval time is not 
always too long, and In particular always lonaer than the 
corresponding time used In the Discrimimation Net technique; If this 
Is the case, obviously the whole thlnq would be a failure. 

The final point concerns the problem of 
storaae on peripheral devices. When a new record Is put Into the 
data base, the question wm ne where to out it outside of main 
memory. As mentioned Dreviously, a reasonable answer Is to put it 
in a "small bucket" with other records that are strongly semantically 
related. Such a bucket may just be the list of records that are 
pointed to by the smallest subindex list on which the record Is also 
located. The fact that this list is the smallest may very well 
Indicate that this suolndex has a stronq semantic meaning, and this 
may be enough to have a reasonable peripneral file organization. 
This point win be Investigated further If the new data base 
organization appears to be competitive with the Discrimination het. 

In summary, it appears to us at the present 
time that this new organization does satisfy the objectives that we 
specified In the first subsection; however, there are still two big 
unknowns that win be resolved only when trie implementation Is 
completed to a fairly advanced point. First, it is not clear what 




















tine improvement we are goina to qet; second, it is not clear now 
much more space wm be required. 

c. Some Additional Details on the Inverted Index 

in subsection b above we qavp the qenerai 
ideas on the new organization; however a lot of details interfere and 
ma<e the actual implementation much more intricate. As the realer may 
know, OLISP has three different types of data; tuples, bags, and 
classes. (be do not mention vectors, whlcn for the present 
discussion are the same type as tuples). we wm guickiv define 
these three data types by their characteristlcs: tuples are the 
n-tuples of ordinary mathematics; baas are its multisets (loosely, 
sets that can have repeated elements); and classes are its classes 
(or sets). For example; (TUPLE A B) is nol equal to (TUPLE b A), 
(BAG A E A) is equal to (BAG A A B) but net to (BAG A B), (CLASS A B 
A) is equal to (CLASS A d). These various equalities or inequalities 
have to appear correctly in the way the indices are chosen. If 
(TUPLE A B) is put into the data ease we need more than just the 
three Indices TUPLE, A, and B because this would not distinguish It 
from (TUPLE B A). Similar remarks apply to baas. The solution 
adopted is as follows (in spirit but not quite in realization). 

(TUPLE A B) corresponds to the indices; 
1TUPLEA 21UPLEB 3TUPLE (this last one 
corresponds to the length of the record). 















(HAG A H) corresponds to toe indices: 


18AGA 1BAUH 3BAG (the lenqth dqain), 

(HAG A A ri) corresponds to the indices: IBACA 
2HAGA 1BAGB 4 8 A G , 

(CLASS A B) corresponds to the Indices: 

CLASSA CLASSB 3CLASS, 

Let us see now how a data retrieval Is 
oerformeo by looking at some examples. Let us consider a request of 
the type (TUPLE A -X C) where -X stands for "any element". This jurt 
means: find all records that have the indices: 1 tupleA 3TUPLEC 

4TUPLE, and is easy to satisfy. In tne same way, the request (HAG A 
-X C) corresponds to all records that nave indices lBAGA laAGC 4BAG. 
Note that in this last case some of the records may also have 2BAGA 
or 2BAGC as a possible index. until now everything nei'-rves nicely. 
Things become slightly more cumbersome with the next example: assume 
the request (TUPLE a C) where .-.-X stands tor "any number of 

arquments," (In otner words this request is matched by (TUPLE A C) as 
well as by (TUPLE A 8 C D E E 3 C).) Then the matchinq records 
certainly have the index 1TUFLEA out their next two indices win oe 
one Of the pairs (2TUPLEC 3TUPLE) , (3TUPLEC 4TUPLE ), and So on. 

The associative retrieval oecomts much more complex. (In the first 
implementation , we will just use 1TUPLLA as the index and foraet 
aoout the fact that there is also a C at the end of the record). 
The similar case behaves very nicely with baqs and classes and does 
not need to be mentioned, 
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In ULISP the "name" of a record, i.e. its 
syntactic content, may be any uIST expression; for example, (TUPLE A 
(TUPLE B C) L)) is a perfectly legal ULISP expression to be stored in 
the data base. This is in fact stored as two records: (TUPLE o C) 
rfhich is, say, record Ri; a n d (TUPLE A : r\ o) where -Hi stands for 
'pointer to Rl". As far as storing these records is concernen, the 
implementation is straightforward; the record (TUPLE a jri dj 
corresponds to the indices; 1TUPLEA 2TUPLE:Hi 3UPI.EU 4TUPLK. The 
difficulties start with the retrieval. How do w e satisfy the 
request: (TUPLE A(TUPLE B *-X)D)? There are two ways to satisfy it. 


We can start by finding all records that match (TUPLE B -X), call 
them RX1 RX2 RX3 ..., then find wnich of the records (TUPLE A :rxi D) 
(TUPLE A :RX2 D) ... exist in the data base. Ur, we mey try to 
directly find all records that match (TUPLE A -Y D) and intersect the 
records that replace -Y with the list (KXl RX2 ...). i n our 
preliminary implementation, we use the second scheme: we find all 
records matchina (TUPLE A ^Y D) and, it there are not too many of 
them, we give all of them to the "pattern matcher" which will 
eliminate the nonmatchlnq ones. If there are too many, we start all 
o r er and try tne first scheme. The reason why we chose this 
implementation is that it was much easier to code and debuq and we 
wanted something running fast to test the ideas; however, it is clear 
that this Implementation is costly both in time and in space (by the 
unnecessary "conses" it generates) and if the new data base ideas 
loo< promising, this will have to be improved upon. 
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Finally# the last level of difficulty is 


caused py i totally different type of retrieval which is possible in 
QLIbP. This can be called the pattern retrieval: a user rray as< to 
find all patterns that match a given datum. For example, one miaht 
ask to find all patterns matching (TUPLE A B) and would expect to re 
given back records like (TUPLE A b), (TUPLE A -X), (TUPLE*-* B), and 
(TUPLE -X *-*-Y) • we will refer to this as the pattern retrieval; this 
turns out to be the most difficult data base operation to implement. 
Let us try to explain how we attempted to solve this problem. in 
the example above, we are given tne data (TUPLE A P); what are the 
possible indices of the matching patterns? A matching pattern could 
have 1TUPLEA but may not: it could have 2TUPLEB but it may not; it 
could have 3TUPLF but again it may not. In fact, at this point it 
looks as though we do not know anything about tne patterns. Tnat is 
not quite true, luckily enough; it the pattern does not have 1TUPLEA 
as an index it must have something like llU r .,L«-X or 1 TUPLE--*. If 
it has lTl'PLEA or 1 TUPLE*-X and does not nave 2TUPLEH, it must have 
2TUPLE-X or 2TUPLE«-*-X. However, if it has 1TUPLE--X, then it may 
very well have nothing else, or rather it may very well have only 

2TUPLE or 3TUPLE and so on. The algorithm to solve this simple 
pattern retrieval is already quite cumbersome; it becomes even more 
cumbersome when treating the following one. 

Let us assume that we are uiven the following 
data and we are lookinq for the matching patterns: (TUPLE A(TUPLE B 
C)U), we then have the same problems as with the last example, Pius 
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a new one. We nave to find all tne patterns that match (TUPLE d C) 
and add that level of comoiexit/ to the algorithm. Here again two 
strategies tray be taken, we may first do the pattern retrieval for 
(TUPLE B C) and then do the Pattern retrievals for the things ii Ke 
(TUPLE A :px 0); this is conceptual1y simoie Dut generates a lot of 
useless records RX and# therefore# a lot of useless pattern retrieval 
attempts. Or we may start py doing a pattern retrieval 
corresponding to (TUPLE A :h- D) where : R- corresponds to "any record 
with a <-Y or X in it" and then intersect the resulting list of :R- 
with the records matching (TUPLE o C). The latter alternative is 
essentially the attitude w e are taking (with the same restricted 
implementation mentioned in the data retrieval part above). Adoption 
of thii strategy causes storage of a new record, such as (TUPLE A 
(TUPLE B -X)), to qenerate the indices 1 TUPLEA 2TMpLE:R<- 2TUPLE:H1 
3TUPLL. The record Hi, which is the name of (TUPLE B *-X) has the 
indices: llUPLEb 2TUPLE-X 3TUPLE. 

There are, of course, a lot more details to 
the implementation than those just given. However# these details 
already give the reader a aeneral feeling ot the difficulties 
involved which did not appear in the general ideas mentioned in 
subsection b. it is important to know tnat these difficulties are 
Present because they are the main cause of time and space 
inefficiencies. It is the solution of these details that will 
allow or disallow the whole idea to be a viable one. 
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4 


Maint enance 


Maintenance produces no milestones but is crucial to 
the utility of a developing system, we continually try to Keep QlISP 
as current as possible; this includes fixing buqs reasonably dUjCklY 
after they are found and tryiny to minimize the lag between the 
arrival of a new version of LISP and the appearance of the matching 
QLISP. While this task, is nominal in extent, its requirements are 
often unpredictable and, since it can occasionally take up to half of 
one person's time (averaged over a month), it can sometimes have a 
noticeable effect on the proqress of developmental tasks, however, 
the Importance of this task leads us to give it a high priority most 
of the time. 


we provide a fast turnaround on repairing buqs in the 
system and providing minor extensions. Users need wait only a couple 
of hours (and never more than 24) to continue using QLISP without 
needing to proqram around the flaws they discover. A high priority 
has also been given tne release of a new QLISP to go along with every 
ne w version of LISP, so that QLISP users may begin using the new LISP 
as soon as possible after its arrival. Under normal circumstances, 
the job of loading a new QLISP and performing various associated 
ancillary tasks would take an hour or less* but in our case this time 
is sometimes increased to as much as several days by two special 
circumstances. First of all, the parts of QLISP directly concerned 


with its interface to LISP are much more sensitive to details of the 














liner structjre 01 I*<ItRLl5P t.ian Tjst otner LISP programs. For 
example* the functions in p L 1 5 P tnat reoiace parts of tne compiler 
ust nav e tre sa-e numc e r of arguments (of the sarr e nanes) as t* e 
standard functions uhey replace. since I'.rKPUSP is also in a 
continual state of development, suer details win cnanqe from tirr e to 
ti-’c. Also# new versions of LISP frequently have now features 
adapted to (and often .rotlvated oy) the constantly increasing 
requirements of LLISP; we try to use the new features rignt a * a v, but 
if testing reveals they 10 not worx# »e Tust patch around them until 
the arrival of the next version of LISP. The Processes of removlna 
old Patches, testinq the resultant ISP, and Dossloly installing new 
pitches can considerately extend the tine required to Dring up a new 
OLISP. 


* e put a high value or, u p w a r a comoatloility; we 
always endeavor t 0 restrict our changes to tne <inos that extend tne 
language without invalidating programs a user nay already have. 
Success at this endeavor Is required for us to retain any given user 
for any considerable lengtn of time, Althougn tnls does not compete 
for Tucn of our time with tne other aspects of the QLISP effort, it 
is something we rust always Keep in mind as we change the UL1SP 


system. 

wish. 


Unfortunately, w e cannot always maintain the continuity w e 
Sometimes overriding considerations dictate a change in the 


user Interface reguirina a change in OLISp programs. A program 
can produce correct results by virtue of a oug which masks a hug in 
viLISP and is masked by it. ^nen the ULISP Dug Is fixed, the program 
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bug takes its toll. We try to minimize those changes that are not 
upwardly compatible, but they unfortunately cannot be eliminated, 

5. Summary 

We have now described how we have directed the part 
of this project devoted to ULISP, Major results are in the areas of 
compilation of user programs# a unification pattern matcher# and a 
new organization of the data storage mechanism, while we also have 
continued an onqolng task of routine maintenance. In our 

undertakings# we have always kept sight of a higher goal: that of 
providing a rellahle# dependable programming system that permits 
users to concentrate on tneir own programs with minimal distraction 
from buqs or other awkward characteristics ULISP may have. 

C. Hardware Work 

1. Interface Equipment 

The interface configuration of th» oresent CBC svstem 
is shown in Fiaure 56. The general approach is to use the local IMP 
and PDP-11 simply as an f/Q processor for the PPP- 10 , ail 

calculation and control »i\\ reside in the PDP-K, while the PDP-11 
takes care of the localized real-time commands and control of thp 
peripherals. All PDP-11 interfaces arp implemented with standard DEC 
modules and hardware. The Analoa-to-Diqital converter is a 


















PDP-10 



FIGURE 56 INTERFACE CONFIGURATION 
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multiplexed 12-blt device fabricated of standard nodules. The 
Dibital-to-Analoa converters (also 1?-blt) Drovlde analoa x-y 
position commands to the ranae finder. 


In use, then, the PDP-11 does the foilowinq. When a 
LISP orooram In the PDP-10 grants to move the camera, It sends to the 
PDP -11 the desired pan and tilt anolesj th e PDP -11 then starts the 
pan/tllt head rnovlno in the proper ilrectton and, via the A/D 
converter, monitors Its Dosltion. When either anole reaches the 
desired position the rotation is stopped, and when both ancles are 
stopped a sianal is sent to the PDP-10 to so sionlfv. 




















If a L 5P nrooran desires a ranoe ’"Pasureirent. it 
sends the desired anaular positions to the PDP-11. The PDP -11 nasses 
these values to the D/A converters. Since the laser deflection 
equipment has no Position feedback# the PDP-11 merelv waits an 
appropriate lenqth of tl ie and then starts the A/D readlna the analoa 
value (Proportional to the ranqe at that Dointl from the ranae 
finder. That value Is sent back to the pdp-io. 

Provision has been made but not yet implemented to 
add control of the TV camera's Iris, focus, zoom, and color wheels, 
and also to a^d communication with the VTP -100 word recoan111 or 
machine. 

2, Laser Ranae Finder: Random Variation of the Ranae 

Measurements Arisino From Imperfections of the Phase 
Measurlna Equipment 

a. Backoround 

In a previous report H7] a discussion was 
qlven of random variations of vanqe measurement arlslnq from the 
finite number of. quanta received by the photomultiplier tube durinn 
the time interval of a measurement. This is certainly the primary 
cause of rang-« "noise," hut it is not the only source; some of the 
observed variability can be attributed to the ohase measurlna 
equipment., and it is the purpose of the followina Invest 1 oat ion to 



















providp data from which one may estimate that component of the 
observed variability. This is not to imply that the Phase mpasurlna 
equipment is defective either as functioning hardware or in design# 
nut rather to recoonize that the dynamic range over which 
measurements are beino taken is well outside the desion for the 
precision we require. in scanning a fl*ld of view that includes an 
oblect such as an air compressor# the innut siaoal to the phase meter 
may cover a dynamic ranae of 105 dR, made up as follows: 


Variation due to inverse square law 18 ; 1 04 

Variation in reflection coefficient ?0 : 1 29 dR 

Specular reflection 20 : i 28 dR 

Surface orientation 20 : 1 2f> dR 

where the maximum sianal comes from a specular reflection from a 
surface close in# and the weakest sianal from a low reflectivity 
surface that is almost tanaential to the scanning beam and at maximum 
range, 


Fiqure 57# redrawn from the HP data .heet, 
shows the maanitude of the offset error to be expected in the output 
of the phase m>ter in relation to the dynamic ranae of th* signal 
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WORST PHASE ERROR — degrees 



















beina measured. This is a displacement error* not a random variation 
error ("noise" on the signal), and by implication is independent of 
the sianal level, which from the nature of the device is rather 
unlikely. 


b. Fxperimentai Observations 

Tests for phase constancy were made usina the 
followinc oieces of equipment made by Hewlett Packard, and believed 
to be in normal ooeratina condition: 

8 6 01 A Slanai Generator 1 9^9 • 

R 407 * 1 Network Analyzer * 959 . 00227 

® 4, 2 A Phase-Magnitude Disnlav # 968 - 00276 

As an initial check the laser ranger was used 
to take 300 ranqe measurements of a point on a table top about 8 feet 
away--this represents an average level siqnal. The mean ranae was 
96.88 inches, and the standard deviation was 0,607 inches. The laser 
had not been tuned for maximum power, and there was nothing unusual 
about the situation in qeneral, so that it seems fair to describe the 
equipment as beinq in normal condition. 

At this point, the two input leads comino 
from the photomultiplier and reference beam photodiode were removed 
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from the test and reference inputs to the Phase meter# and all of thp 
subsequent measurements to be discussed were made usim slqnals from 
tne «601A Sianal Generator, the sinnlp output from which was 
converted to dual outnut by means of the oower srlitfer 11651 - 
60001. In oPeratinq the laser ranoer it wgs observed that tne two 
vertical slide attenuators on the Phase Meter, marked Display 
Reference dp# and which presumably affect only the display circuitry# 
nevertheless do interact with the measitrino equipment and in 
particular affect the phase displacement error associated with 
limitino shown in Fioure 57. (When scanninq over a oray-scale step 
Chart at constant distance, tne ranoe readino snowed minimum offset 
error when the attenuators were at. +10 do and +?dH.l 

Test No. 1 Identical Sionals to Both Channels# as shown 

in Fiqure 5R. 

Input frequency R .000 MHz Bandwidth 100 Hz 

loo measurements at 30 millisecond intervals 


Sianal 

Ranoe 

Standard 

Level 

(inches ) 

Deviation 

-10 dB 

115.04 

0.09b 

-20 dB 

114.56 

0.07R 

-30 dB 

114.53 

0.056 


-40 d0 


114.96 


0.126 



























8601 A 

SIGNAL GENERATOR 
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FIGURE 58 TEST 1: IDENTICAL SIGNALS IN BOTH CHANNELS 

The -40 dH level corresponds to an Input sional of 3 millivolts. One 
inch in ranae corresponds to approximately one degree in phase shift. 
Thus, for relatively hiah input levels, identical in both -hanr.els, 
the measurements are essentially noise freer how e ver, the 
displacement of the zero bv 0.5 inch for only 10 dR chance in level 
is significant, and in the light of previous experience can probahlv 
be attributed to effects in the reference channel, it is desirable 
to adlust the reference level to read close to mid-scale on meter, 
and not merely keep it. within the Permissible ranae marked on the 
dial, 

Test No. 7 Reference Channel 40 dB Above Test Channel, 

as shown in Figure 59. 


100 Neasurements at .30 millisecond intervals. 




















The reference level was maintained close to mid-scale by adjustment 


of the three position attenuator o the Phase Meter panel. This 
naturally caused displacement of the zero because of small 


differences o 

wlrind lenoths within 

the attenuate-, 

but our orimarv 

interest was in 

measuring the "noise 

" level? i . e . 

, toe standard 

deviation of the 

measurements. 



Test 

Equivalent 

Ranqe 

Standard 

I evel 

Value 

finches! 

heviation 

-20 dR 

30 mv 

114.70 

0.009 

-30 dR 

1 0 mv 

114,70 

0.097 

-50 dB 

1 mv 

11 A.28 

0.103 

-60 dB 

300 uv 

122.56 

0.361 

-70 dR 

100 UV 

124.R0 

0.758 

-80 dB 

3 0 UV 

120,19 

4.352 
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FIGURE 59 TEST 2: NORMAL CONDITION OF USE 































From the above measurements It is evident 
that random variations due to the measuring eouiprent itself are 
significant for test levels below 100 microvolts. Tt is difficult to 
arrange for anv test scene that includes the comoressor to cover a 
range of less than 70-90 dB, Moreover, If the photc m u111nl 1 er gain 
is increased to raise the low-level signals until thev cause the 
Phase meter to operate in the low-noise rang*, then the preamplifier 
will go into nonlinear operation on the specular reflections and will 
produce substantial positional errors. Tt is perbaos worth pointing 
out that the time delay of the signal created by a nhntomultipiier is 
not insianlf icant in tho present context, and that this time delay is 
a function of the rower sunnly voltage, which is used as the gain 
control. Fluctuations of the cower sudpiv voltage therefore modulate 
the phase difference between the test and reference channels, and are 
thus a further source of "noise" on the range measurement. 

h. Artificial Intelligence Center Computer Facility 

1. Ma|or Accomnlishments 

Figure 60 shows the current POP-10 TFMFX computer 
hardware block diagram. The two malor changes during this contract 
year were the total replacement of all existing main memory (which 
was very unreliable! with a new 256, ono-wrrd mpmory and the 
replacement of a five-drive disk system (which was also very 
unreliable) with a used nine-drive disk system that is considerably 


better. 
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The ooer atina system software has underdone a malor 


restructuring In order to provide moi« fPU cycles to the users at a 
small loss in interactive response. The current hardware/software 
conflouratior now routinely delivers R5 to 90 percent of all CPU 
cycles to the user durina perils of normal load. 

?. Portents for the Future 

The current comouter facility is heavily looded for 
about R0 hours per weetr. Fost oi our research proarams are limited 
by our computer power and address soace. Both of these areas are 
beina investigated thorounhiy in order to provide the computing 
service that the research staff needs. 
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APPENDIX 


CHOUSING A TASK AREA FOR THE CBC PROJECT 


A. Criteria 

Given that our CBC system is to be an expert consultant about 
some complex electromechanical eauipment of interest to the non, 
there is still the problem ot selecting some specific class of 
equipment from amonq thousands of military items, we have acquainted 
ourselves with a few of the possible candidates, and we have looted 
into some of them quite thorouqhiy. Our prime consideration has 
been to select an Item or class of Items that meets certain criteria 
that we have adopted. We shall discuss these criteria first and then 
give some details about the equipment we nave considered, 

1. Relevance to noD 

The equipment should be of a tvoe for which Increased 
automation of maintenance win lead to substantial cost reduction 
and/or Improved operation. 
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Amenability to the Use of Automation v ethnrts 


The eaulnment and its normal use should be such that 
computprized maintenance methods (if theV can be developed! are 
feasible in an operational setting. It would be esoeclaliv desirable 
if there were already beainnina attempts to use computer1 red 
maintenance methods on the eouipment. 

3, Suitability for tne Kesearc* Goals 

The equipment must he such that the research 
Performed usina It leads to broadly useful results rather than 
results of interest for that equipment onlv. 

4, Fxperimental Convenience 

The equipment must he of a scale and complexity that 
Permits us to experiment with it easily. It should not reduire 
Inordinately elaborate test oear nor should It be too difficult or 
expensive to obtain, 

5, Availability of Expertise 

There should be a wealth of information about the 

equipment in the form of manuals, drawinos, and nearby and accessible 


human experts 















B. 


Candidate Equipment 


1. Genera 1 

be have considered quite a fe* equinment items and 
have looked into four of them in some detail, flmonq the Items we 
have only superficially thouqht about and rejected for various 
reasons before further study are* automatic manipulators (such as 
the Unimate arm), instruments such as qyr 0 s C ores, refrioeration 
systems, diesel enqines for tanks and trucks, control-rod actuation 
mechanisms for nuclear reactors, lame pneumatic compressor systems, 
torpedoes, and automatic transmissions. 

The four items that survived initial sereenina are 
Jeep enqines, hydraulic flight control actuators, Generator sets, and 
the steerinn mount for the awg- 10 radar for the A4 aircraft. We will 
comment on each of th*se in toe next sections. We expect to select 
one of these items shortly. 

2. Jeep Fnolnes 

Jeep enqines are attractive because there is already 
an automatic system currently beino field tested for d'laonoslno them. 
This system is the 6TE/ICF belnq develooed under contract to TACOM 
[46], We have had Initial conversations with personnel from TACOM, 
Frankford Army Arsenal, and PCA ./ho have been involved in thl* 


























orolect, and these have been on the vhole quite encouraqinn. We 
would he able to build on the existlm technoloav, addina more 
flexible corrmunirations and more Dowerful diaanostir procedures. 
There is a good chance that we would be able to obtain a copy of the 
ATFJ/ICF system. 

A leer would be reasonably acceptable from the 
experimental convenience point of view, except that running the 
enaine inside wwpuld reauire special venting of the exhaust. It 
scores well on all of toe other criteria. 

3. Hydraulic Flioht Control Actuators 

On every commercial and milltarv let airplane there 
are several hydraulic actuators movinq fliqht control surfaces such 
as elevators and ailerons. These are moderately complex devices that 
are nontrivial to troubleshoot. 

We have not vet received official permission to visit 
the Naval Air Rework- Facility at Alameda Naval Air Station to talk to 
the appropriate maintenance Deoole# hut in the meantime w? visited 
the maintenance headauarters of United Airlines. There we learned of 
the variety of different actuators for commercial aircraft and 
something about the maintenance problems involved. 
















Our main concern about workiro with one of these 
actuators is experimental convenience. Some of these conslderattons 
include the skin irrjtatinq Drooerties of the hydraulic fluid used in 
commercial actuators (military actuators are said to be better In 
this reaard), the problem of fluid atomizin'., and the need for 
several kinds of special sensors--e.o,, flow meters, manometers, 
force transducers, position transducers. 

There Is a possibility that we could use United Air 
Lines' exisMnn facilities, and this would solve some of the 
experimental, problems. 

4. Generator 5ets 

DoO uses a wide variety of aenerator sets ranqina 
from small portable field units to 1 a r o e Dower systems. All of the 
newer ones are either turbine or diesel n 0 wered. A complete 
medium-sized system such as turbine powered fmij -30 (30 kN) involves 
complex electronic, electrical, and mechanical subsystems. 

We visited the aenerator llasion office at McClellan 
Air Force Base in California to learn more about the characteristies 
of qenerators and their maintenance problems. (jur conclusions are 
that a system could be chosen of the rioht level of difficulty, but 
there are some severe experimental convenience problems: they are 
veiv noisy. 
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The AwG-10 Steerable Ra Jar Mount 


Vvp discussed the steerable radar mount for the AWG-io 
radar uspd in the A4 aircraft at Miramar Naval Ajr Station. This is 
a comnlex device contalnin nrotors and servos to aim the radar 
antenna. It is of about the rioht size ohvsicallv and *ould he 
optimal from an exDerimental convenience ooint of view. 

Kp nave some concern that it miaht not represent a 
maintenance problem of sufficiently aeneral interest* and we have 
also experienced certain difficulties already in aeftina the 
appropriate Navy publications descrihino if. 
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